正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

We release a technical blog on training PocketTTS, our 100M parameter on-device TTS, through drifting, a recent one-step generative objective from Deng et al. Less than 1% WER, high quality and voice cloning. To the best of our knowledge, it's the first speech model, and the first autoregressive model,... show more

kyutai

27,822 subscribers

17,569 次观看 • 3 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

11 条评论

kyutai 的头像

kyutai3 天前

PocketTTS speaks in continuous latents, one every 80 ms, and a small head turns noise into the next latent in a single forward pass. We used to train with LSD, another one-step flow matching loss, but this required Jacobian-vector products at training time. This is not needed anymore with the drifting recipe.

kyutai 的头像

kyutai3 天前

The key ingredient we came up with to make it work is a learned kernel temperature. The full write-up with the recipe and all the ablations is here : Code: Original Drifting paper:

Aïna 的头像

Aïna3 天前

Can I just say this little guy is so cute

LastResort 的头像

LastResort3 天前

@grok which language it supports?

{stc} 的头像

{stc}3 天前

nice work playas

Francisco Carlos Erra 的头像

Francisco Carlos Erra2 天前

🙂

Eman Khoubian 的头像

Eman Khoubian3 天前

less than 1% WER on TTS means a speech recognizer heard the words. it says nothing about whether the cloned voice still sounds right at minute 20, which is where synthetic voices break.

Yazid Janati 的头像

Yazid Janati3 天前

@YentiCollin @samsongvch

Vi VL 的头像

Vi VL3 天前

Есть поддержка realtime?

mdzor 的头像

mdzor3 天前

@LocallyAIApp 🙈

semonic 的头像

semonic3 天前

Awesome i love this project I just wish you guys jad used more and better training data

相关视频

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required. Open-source, lightweight, and incredibly fast. 🧵👇

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required. Open-source, lightweight, and incredibly fast. 🧵👇

kyutai

238,618 次观看 • 8 个月前

Introducing Soniox TTS v2, our most powerful text-to-speech model yet. Soniox TTS v2 brings extraordinary voice quality, expressive control through audio tags, exceptional precision, high-fidelity voice cloning, more than 60 languages, natural language mixing, and low-latency streaming together in one model. Built from the ground up with a new model architecture, audio codec, and highly optimized inference engine, Soniox TTS v2 delivers frontier TTS at just $0.70 per generated hour. Available globally today as tts-rt-v2. Try it out: Turn the sound on and hear what’s possible. 🔊

Introducing Soniox TTS v2, our most powerful text-to-speech model yet. Soniox TTS v2 brings extraordinary voice quality, expressive control through audio tags, exceptional precision, high-fidelity voice cloning, more than 60 languages, natural language mixing, and low-latency streaming together in one model. Built from the ground up with a new model architecture, audio codec, and highly optimized inference engine, Soniox TTS v2 delivers frontier TTS at just $0.70 per generated hour. Available globally today as tts-rt-v2. Try it out: Turn the sound on and hear what’s possible. 🔊

Soniox

666,014 次观看 • 1 个月前

Pocket TTS goes multilingual! Now you can use our 100M-parameter models to generate speech in six languages, fast enough to run real-time without a GPU. We also improved the quality of the English model while keeping the same size. And all of this is open-source.

Pocket TTS goes multilingual! Now you can use our 100M-parameter models to generate speech in six languages, fast enough to run real-time without a GPU. We also improved the quality of the English model while keeping the same size. And all of this is open-source.

kyutai

30,012 次观看 • 5 个月前

We're releasing Paris 2.0, which, to our knowledge, is the world's first decentralized trained video generation model. We benchmarked it against a monolithic model trained on the same data and compute budget, and Paris 2.0 outperformed the monolithic by ~2x on FVD benchmark.

We're releasing Paris 2.0, which, to our knowledge, is the world's first decentralized trained video generation model. We benchmarked it against a monolithic model trained on the same data and compute budget, and Paris 2.0 outperformed the monolithic by ~2x on FVD benchmark.

Bidhan Roy

457,629 次观看 • 4 个月前

[SIGGRAPH Asia '24 (TOG)] Representing Long Volumetric Video with Temporal Gaussian Hierarchy Contributions: • We introduce a novel, efficient, and expressive Temporal Gaussian Hierarchy representation for long volumetric video. To our knowledge, our method is the first approach capable of handling minutes of volumetric video data. • We propose a Compact Appearance Model and a new rasterization implementation to facilitate real-time, high-quality dynamic view synthesis while maintaining a compact size. • We propose a system to efficiently model long volumetric videos for the first time and demonstrate state-of-the-art dynamic view synthesis quality on the Neural3DV [Li et al. 2022], ENeRF-Outdoor [Lin et al. 2022], and MobileStage [Xu et al. 2024b] datasets, while also achieving the best rendering speed with reduced training cost and memory usage.

[SIGGRAPH Asia '24 (TOG)] Representing Long Volumetric Video with Temporal Gaussian Hierarchy Contributions: • We introduce a novel, efficient, and expressive Temporal Gaussian Hierarchy representation for long volumetric video. To our knowledge, our method is the first approach capable of handling minutes of volumetric video data. • We propose a Compact Appearance Model and a new rasterization implementation to facilitate real-time, high-quality dynamic view synthesis while maintaining a compact size. • We propose a system to efficiently model long volumetric videos for the first time and demonstrate state-of-the-art dynamic view synthesis quality on the Neural3DV [Li et al. 2022], ENeRF-Outdoor [Lin et al. 2022], and MobileStage [Xu et al. 2024b] datasets, while also achieving the best rendering speed with reduced training cost and memory usage.

MrNeRF

79,493 次观看 • 1 年前

NEW: Higgs Audio V2 from BosonAI open, unified TTS model w/ voice cloning, beats GPT 4o mini tts and ElevenLabs v2 🔥 > Trained on 10M hours (speech, music, events) > Built on top of Llama 3.2 3B > Works real-time and on edge > Beats GPT-4o-mini-tts, ElevenLabs v2 in prosody & emotion Multi-speaker dialog > Zero-shot voice cloning 🤩 > Available on Hugging Face Kudos to folks at Boson AI for releasing such a brilliant work and all the details around the model! 🤗

NEW: Higgs Audio V2 from BosonAI open, unified TTS model w/ voice cloning, beats GPT 4o mini tts and ElevenLabs v2 🔥 > Trained on 10M hours (speech, music, events) > Built on top of Llama 3.2 3B > Works real-time and on edge > Beats GPT-4o-mini-tts, ElevenLabs v2 in prosody & emotion Multi-speaker dialog > Zero-shot voice cloning 🤩 > Available on Hugging Face Kudos to folks at Boson AI for releasing such a brilliant work and all the details around the model! 🤗

Vaibhav (VB) Srivastav

79,645 次观看 • 1 年前

LETS GOOO! FishSpeech v1.5 - multilingual, zero-shot instant voice cloning, low-latency, open text to speech model 🔥 > Only 500M params > Trained on 1 MILLION hours of audio > Supports 13 languages > Low-latency ( Open model - checkpoints on the hub 🤗 > Best part: Ranks #2 on TTS Arena (preliminary results) Kudos to Fish Audio - truly leading the way for open text to speech research! ⚡

LETS GOOO! FishSpeech v1.5 - multilingual, zero-shot instant voice cloning, low-latency, open text to speech model 🔥 > Only 500M params > Trained on 1 MILLION hours of audio > Supports 13 languages > Low-latency ( Open model - checkpoints on the hub 🤗 > Best part: Ranks #2 on TTS Arena (preliminary results) Kudos to Fish Audio - truly leading the way for open text to speech research! ⚡

Vaibhav (VB) Srivastav

49,150 次观看 • 1 年前

Today, we’re thrilled to unveil the Arcee.ai Foundation Models, a new family of GenAI models designed from the ground up for enterprise reality. The first release—AFM-4.5B—is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost efficiency. In short: enterprise-grade intelligence that can run anywhere—on a smartphone, at the edge, or in the cloud. For a quick taste, you can test AFM-4.5B in our playground and on For a deeper dive into the model’s training pipeline and benchmarks, details are available in our technical blog post. ➡️ Arcee AI playground: ➡️ Together AI playground: ➡️ Launch blog post: ➡️ Technical blog post: PS: As we’ll now focus on our AFM foundation models, and because we love open-source, we're opening up access to our previously closed-source language models. Details in the tech blog post!

Today, we’re thrilled to unveil the Arcee.ai Foundation Models, a new family of GenAI models designed from the ground up for enterprise reality. The first release—AFM-4.5B—is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost efficiency. In short: enterprise-grade intelligence that can run anywhere—on a smartphone, at the edge, or in the cloud. For a quick taste, you can test AFM-4.5B in our playground and on For a deeper dive into the model’s training pipeline and benchmarks, details are available in our technical blog post. ➡️ Arcee AI playground: ➡️ Together AI playground: ➡️ Launch blog post: ➡️ Technical blog post: PS: As we’ll now focus on our AFM foundation models, and because we love open-source, we're opening up access to our previously closed-source language models. Details in the tech blog post!

Arcee.ai

21,934 次观看 • 1 年前

You don't need a GPU to run text-to-speech anymore. Kyutai released Pocket TTS, a 100 million parameter speech model that runs entirely on CPU. How it performs: > Nearly 6x faster than real time on a MacBook CPU > First audio chunk arrives in about 200 milliseconds > Runs on just 2 CPU cores > Supports voice cloning and 6 languages Setup takes one command: pip install pocket-tts Text-to-speech that used to need a GPU or a paid API now runs locally, on a laptop, in near real time.

You don't need a GPU to run text-to-speech anymore. Kyutai released Pocket TTS, a 100 million parameter speech model that runs entirely on CPU. How it performs: > Nearly 6x faster than real time on a MacBook CPU > First audio chunk arrives in about 200 milliseconds > Runs on just 2 CPU cores > Supports voice cloning and 6 languages Setup takes one command: pip install pocket-tts Text-to-speech that used to need a GPU or a paid API now runs locally, on a laptop, in near real time.

Simplifying AI

14,456 次观看 • 1 个月前

.L3Harris says Palantir helped them beat frontier AI models in less than 2 days, at 95% lower cost: "When we fine-tuned open source models trained on our own data, we were able to outperform the frontier models in less than 48 hours." "The cost of our fine-tuned open source model was 95% lower than the frontier models we were using." "AI is a commodity. It's all about the data. We view our data as a corporate asset. It's our unique hard-earned knowledge." "We believe American defense companies should not be a vassal for frontier AI labs, handing over our data and institutional knowledge, hoping to rent back the intelligence it creates." " We own the model, we own the compute, we own the advantage."

.L3Harris says Palantir helped them beat frontier AI models in less than 2 days, at 95% lower cost: "When we fine-tuned open source models trained on our own data, we were able to outperform the frontier models in less than 48 hours." "The cost of our fine-tuned open source model was 95% lower than the frontier models we were using." "AI is a commodity. It's all about the data. We view our data as a corporate asset. It's our unique hard-earned knowledge." "We believe American defense companies should not be a vassal for frontier AI labs, handing over our data and institutional knowledge, hoping to rent back the intelligence it creates." " We own the model, we own the compute, we own the advantage."

Jawwwn

1,105,305 次观看 • 24 天前

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.

Made by Google

28,283 次观看 • 2 年前

We are now entering the next chapter for Quartr Pro. Our generative AI model is trained solely on first-party information from public companies and to always link back to the original sources – making it truly stand out. Contact our team via the link in our bio to try it out!

We are now entering the next chapter for Quartr Pro. Our generative AI model is trained solely on first-party information from public companies and to always link back to the original sources – making it truly stand out. Contact our team via the link in our bio to try it out!

Quartr

29,011 次观看 • 2 年前

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,738 次观看 • 2 年前

We've released the code for LegoGPT. This autoregressive model generates physically stable and buildable designs from text prompts, by integrating physics laws and assembly constraints into LLM training and inference. This work is led by PhD students Ava Pun, Kangle Deng, Ruixuan Liu, and in collaboration with CMU faculty Changliu Liu and Deva Ramanan. LegoGPT is a small first step towards the ultimate goal of generative manufacturing of physical objects. Our implementation is limited to 20x20x20 dimensions, 21 object categories, and simple brick types, but we are working on scaling it up! Code: Website: Demo:

We've released the code for LegoGPT. This autoregressive model generates physically stable and buildable designs from text prompts, by integrating physics laws and assembly constraints into LLM training and inference. This work is led by PhD students Ava Pun, Kangle Deng, Ruixuan Liu, and in collaboration with CMU faculty Changliu Liu and Deva Ramanan. LegoGPT is a small first step towards the ultimate goal of generative manufacturing of physical objects. Our implementation is limited to 20x20x20 dimensions, 21 object categories, and simple brick types, but we are working on scaling it up! Code: Website: Demo:

Jun-Yan Zhu

38,913 次观看 • 1 年前

🚀 VoxCPM 2 is live! 🎉 Another open-source AI #TTS model from China — and one that stands shoulder to shoulder with Qwen3-TTS, while bringing everything into a single unified model. After rapid iterations from V1 (zero-shot cloning) to V1.5 (long-form + fine-tuning), #VoxCPM has consistently pushed quality and usability forward. Now, VoxCPM 2 takes it further: 🔹30+ languages — truly global, truly local. 🔹Infinite voice design — type it, hear it, control it. From a whisper to a booming cinematic voice. 🔹Studio-grade audio — 48kHz ultra-high fidelity with emotional depth 🔹Diffusion-Autoregressive cloning — preserves more acoustic and emotional detail than token-based models like Qwen3-TTS 💡 Big shoutout to Grok — used your multi-image video magic for our launch demo. It’s scarily good at keeping visuals consistent across shots. Elon Elon Musk, this one’s for you. 😉 Check the demo & start cloning your dream voice: 🌐 Hugging Face Space: 🤗 Hugging Face Model: 🤖 ModelScope Model: 💻 GitHub： #TTS #AI #VoiceCloning #GrokImagine #ElonMusk #OpenBMB #VoxCPM

🚀 VoxCPM 2 is live! 🎉 Another open-source AI #TTS model from China — and one that stands shoulder to shoulder with Qwen3-TTS, while bringing everything into a single unified model. After rapid iterations from V1 (zero-shot cloning) to V1.5 (long-form + fine-tuning), #VoxCPM has consistently pushed quality and usability forward. Now, VoxCPM 2 takes it further: 🔹30+ languages — truly global, truly local. 🔹Infinite voice design — type it, hear it, control it. From a whisper to a booming cinematic voice. 🔹Studio-grade audio — 48kHz ultra-high fidelity with emotional depth 🔹Diffusion-Autoregressive cloning — preserves more acoustic and emotional detail than token-based models like Qwen3-TTS 💡 Big shoutout to Grok — used your multi-image video magic for our launch demo. It’s scarily good at keeping visuals consistent across shots. Elon Elon Musk, this one’s for you. 😉 Check the demo & start cloning your dream voice: 🌐 Hugging Face Space: 🤗 Hugging Face Model: 🤖 ModelScope Model: 💻 GitHub： #TTS #AI #VoiceCloning #GrokImagine #ElonMusk #OpenBMB #VoxCPM

OpenBMB

560,058 次观看 • 6 个月前

We are excited to share a research preview of our generative agent. The agent is being trained to solve the hardest tasks in 3D and beyond, using only keyboard and mouse actions. Join the waitlist: Our agent app runs on Windows or Mac, either locally or with one-click setup for a Windows VM. It’s still early days, but this paves the way for production-level workflows for the first time ever. Blog:

We are excited to share a research preview of our generative agent. The agent is being trained to solve the hardest tasks in 3D and beyond, using only keyboard and mouse actions. Join the waitlist: Our agent app runs on Windows or Mac, either locally or with one-click setup for a Windows VM. It’s still early days, but this paves the way for production-level workflows for the first time ever. Blog:

Common Sense Machines

155,199 次观看 • 1 年前

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.

Freesolo

29,076 次观看 • 2 个月前

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,336 次观看 • 2 年前

TEDRA Text-based Editing of Dynamic and Photoreal Actors discuss: Over the past years, significant progress has been made in creating photorealistic and drivable 3D avatars solely from videos of real humans. However, a core remaining challenge is the fine-grained and user-friendly editing of clothing styles by means of textual descriptions. To this end, we present TEDRA, the first method allowing text-based edits of an avatar, which maintains the avatar's high fidelity, space-time coherency, as well as dynamics, and enables skeletal pose and view control. We begin by training a model to create a controllable and high-fidelity digital replica of the real actor. Next, we personalize a pretrained generative diffusion model by fine-tuning it on various frames of the real character captured from different camera angles, ensuring the digital representation faithfully captures the dynamics and movements of the real person. This two-stage process lays the foundation for our approach to dynamic human avatar editing. Utilizing this personalized diffusion model, we modify the dynamic avatar based on a provided text prompt using our Personalized Normal Aligned Score Distillation Sampling (PNA-SDS) within a model-based guidance framework. Additionally, we propose a time step annealing strategy to ensure high-quality edits. Our results demonstrate a clear improvement over prior work in functionality and visual quality.

AK

17,302 次观看 • 2 年前

Marvis-TTS-v0.2 is here 🚀 A local first TTS model capable of realtime performance even on older iPhones that Lucas Newman and I built. What’s new: ✨ Blazing fast — 100M (tiny) & 250M parameter models 🌍 Multilingual — English, French, German 🎭 Enhanced voice cloning — More natural & expressive ⚡ Long-form generation — Up to 90 seconds (4x improvement) Get started today: > pip install -U mlx-audio

Marvis-TTS-v0.2 is here 🚀 A local first TTS model capable of realtime performance even on older iPhones that Lucas Newman and I built. What’s new: ✨ Blazing fast — 100M (tiny) & 250M parameter models 🌍 Multilingual — English, French, German 🎭 Enhanced voice cloning — More natural & expressive ⚡ Long-form generation — Up to 90 seconds (4x improvement) Get started today: > pip install -U mlx-audio

Prince Canuma

118,556 次观看 • 11 个月前