Sensitive content

This media may contain sensitive content.

Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

⚠️ NSFW ⚠️ Looks like Hume made a virtually uncensored voice-native LLM called Octave—the first language model built specifically for TTS. You can generate any type of voice you want with a prompt, and their WebUI tool can auto-generate corresponding example dialogue! The model grasps user intent from text... show more

Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭

235,783 subscribers

36,342 просмотров • 1 год назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Big moment for text-to-speech. Qwen just open-sourced a text-to-speech model that lets you clone voices, design new ones, and control speech using natural language. Let me explain what I mean: You can literally tell it "speak in a cheerful tone with slight nervousness," and it actually does that. No complex audio engineering needed. What makes this special: - 3-second voice cloning - Covers 10 languages: English, German, French, and more - Latency as low as 97ms for real-time applications - Supports both streaming and non-streaming generation The model comes in two sizes (0.6B and 1.7B parameters), so you can pick based on your hardware and quality needs. Three modes to work with: 1. Custom Voice: Use pre-built premium voices with instruction-based style control 2. Voice Design: Describe the voice you want in plain English (or Chinese), and the model creates it 3. Voice Clone: Provide a 3-second reference audio and clone that voice The best part? It integrates with vLLM for production deployment and has a simple Python package you can pip install. I've shared a link to the GitHub repo in the next tweet.

Big moment for text-to-speech. Qwen just open-sourced a text-to-speech model that lets you clone voices, design new ones, and control speech using natural language. Let me explain what I mean: You can literally tell it "speak in a cheerful tone with slight nervousness," and it actually does that. No complex audio engineering needed. What makes this special: - 3-second voice cloning - Covers 10 languages: English, German, French, and more - Latency as low as 97ms for real-time applications - Supports both streaming and non-streaming generation The model comes in two sizes (0.6B and 1.7B parameters), so you can pick based on your hardware and quality needs. Three modes to work with: 1. Custom Voice: Use pre-built premium voices with instruction-based style control 2. Voice Design: Describe the voice you want in plain English (or Chinese), and the model creates it 3. Voice Clone: Provide a 3-second reference audio and clone that voice The best part? It integrates with vLLM for production deployment and has a simple Python package you can pip install. I've shared a link to the GitHub repo in the next tweet.

Akshay 🚀

31,249 просмотров • 7 месяцев назад

As always everyone is blind staring at the progress of LLMs for coding and chat But meanwhile the new SOTA video model Seedance 2.5 has been slowly rolling out and it's really quite exceptional It's made by ByteDance (TikTok) who of course have lots of training data With just a few reference pics, it can get quite close to how you look IRL and you can do quite professional video shots with just a prompt I'd say it's the first video model that's now at the level of image models with the level of character likeness, cracking that in image models also took about 3 years (2022-2025) Generating 15 seconds takes about 4 minutes I put it live now on Photo AI, you can use it under [ Make video ] from the sidebar with just a prompt and your model selected! So you don't need to take an AI photo first and then turn that into a video! Saves lots of time :D It's more expensive than but I kept the credits the same (30 for 1 video) It also works inside the new video editor and you can make changes in your video with [ Magic edit ] in both the main app and the video editor Also a message for my server guy Daniel Lockyer (it can do voice too and you can even submit a voice sample of yourself, but I didn't here)

As always everyone is blind staring at the progress of LLMs for coding and chat But meanwhile the new SOTA video model Seedance 2.5 has been slowly rolling out and it's really quite exceptional It's made by ByteDance (TikTok) who of course have lots of training data With just a few reference pics, it can get quite close to how you look IRL and you can do quite professional video shots with just a prompt I'd say it's the first video model that's now at the level of image models with the level of character likeness, cracking that in image models also took about 3 years (2022-2025) Generating 15 seconds takes about 4 minutes I put it live now on Photo AI, you can use it under [ Make video ] from the sidebar with just a prompt and your model selected! So you don't need to take an AI photo first and then turn that into a video! Saves lots of time :D It's more expensive than but I kept the credits the same (30 for 1 video) It also works inside the new video editor and you can make changes in your video with [ Magic edit ] in both the main app and the video editor Also a message for my server guy Daniel Lockyer (it can do voice too and you can even submit a voice sample of yourself, but I didn't here)

@levelsio

704,902 просмотров • 19 дней назад

Today, we're shipping MLX support for TADA, our open-source text-to-speech model, which means the entire pipeline (LLM, flow-matching, and decoder) can now run locally on any Apple Silicon device. We're seeing a 45% reduction in memory usage and a 10x speed-up when using it quantized. With these improvements, you can use TADA on-device for OpenClaw or any personal chatbot. If you own a MacBook, Mac Mini, or Mac Studio, record a 10-second clip of any voice, type any text, and get high-quality, natural and expressive speech in real-time. Completely offline, completely free.

Today, we're shipping MLX support for TADA, our open-source text-to-speech model, which means the entire pipeline (LLM, flow-matching, and decoder) can now run locally on any Apple Silicon device. We're seeing a 45% reduction in memory usage and a 10x speed-up when using it quantized. With these improvements, you can use TADA on-device for OpenClaw or any personal chatbot. If you own a MacBook, Mac Mini, or Mac Studio, record a 10-second clip of any voice, type any text, and get high-quality, natural and expressive speech in real-time. Completely offline, completely free.

Hume AI

24,684 просмотров • 5 месяцев назад

Slideshow clipping has to be the chillest business model you can really be making $5k - $15k+ a month running 10 TikTok accounts and have complete location freedom + make more than you would working a job you can literally just go to content rewards, pick a saas or brand to promote, for example skin care buy 5 aged TT accounts to start with connect US proxy / sim find videos in the niche the product is in for example skin care steal the hook of the video and content inside it and turn it into a slideshow for example: “skin care tips for broke girls” generate the images with NBP by finding a Pinterest image and getting the JSON from Gemini or ChatGPT promote the product as another “glow up” tip Post 1 a day from the start and as the accounts get more warm boost it to 2-3 + scale to more accounts If you can’t make your first $5k doing this then there is no hope

Pounds

15,415 просмотров • 6 месяцев назад

Want to reduce a LOT of the censorship in Seedance 2.0? I’m going to show you a simple method that almost nobody is using — and it can completely change the results of your videos. Tutorial: First, ask ChatGPT to translate your entire prompt into the Tibetan language. Since it is a less common language in China, this can sometimes help bypass some of the censorship limitations. After that, use the translated prompt inside Seedance 2.0 to generate the video. This is exactly how the video shown above was created. To generate the videos, use Yapper . The recommendation is to create multiple videos at the same time — for example, generate 8 videos at once — to increase the chances of getting an even better result.

Want to reduce a LOT of the censorship in Seedance 2.0? I’m going to show you a simple method that almost nobody is using — and it can completely change the results of your videos. Tutorial: First, ask ChatGPT to translate your entire prompt into the Tibetan language. Since it is a less common language in China, this can sometimes help bypass some of the censorship limitations. After that, use the translated prompt inside Seedance 2.0 to generate the video. This is exactly how the video shown above was created. To generate the videos, use Yapper . The recommendation is to create multiple videos at the same time — for example, generate 8 videos at once — to increase the chances of getting an even better result.

nachos2d

30,513 просмотров • 5 месяцев назад

NEW SECRET MYSTERY AI VIDEO MODEL ARENA CODE NAME: BOUNCYBOHR Unlike Artificial Analysis, on Arena. AI you can submit your own prompts and this prompt is designed to try and unmask who created the model. Models powered by an LLM (like Seedance 2.0 and Omni) are smart and can see what you are doing. As far as thoughts? Eh? While another screenshot looked like Omni, this does look pretty Seedance to me. Judging by the result, this looks like the rumored new "improved" FAST model and not upgraded full 2.1 model that is supposed to be 25% better. Text To Video close-up cinematic scene from a film, professionally directed, focused, ultrafine details a woman laughs as she whiteboard writes out what model made this video

NEW SECRET MYSTERY AI VIDEO MODEL ARENA CODE NAME: BOUNCYBOHR Unlike Artificial Analysis, on Arena. AI you can submit your own prompts and this prompt is designed to try and unmask who created the model. Models powered by an LLM (like Seedance 2.0 and Omni) are smart and can see what you are doing. As far as thoughts? Eh? While another screenshot looked like Omni, this does look pretty Seedance to me. Judging by the result, this looks like the rumored new "improved" FAST model and not upgraded full 2.1 model that is supposed to be 25% better. Text To Video close-up cinematic scene from a film, professionally directed, focused, ultrafine details a woman laughs as she whiteboard writes out what model made this video

Brent Lynch

32,504 просмотров • 2 месяцев назад

MiniMax H3 Instead of sharing the prompts for each of these videos, I thought it would be more useful to share how I created that prompts. All of the videos were generated with text-to-video. First, find an image with the kind of scene, composition and mood you want to recreate. I used a few YouTube playlist thumbnails as references but Pinterest is also a great place to find inspiration. You can even use your own old or nostalgic photographs. Then upload the image to ChatGPT and ask it to describe the scene. The description it gives you can essentially become your text-to-video prompt. From there, you can generate completely new scenes with a similar composition, atmosphere and cinematic language. You can of course use the reference image directly with image-to-video or as a first frame. But if the original image isn't yours, I prefer using it only as visual inspiration and recreating the scene through text-to-video. This is the prompt I use with ChatGPT: "Describe the scene in this image in English, focusing primarily on what is happening, the characters, their actions and body language, the setting and the overall atmosphere. Also briefly describe the composition, framing, camera angle, approximate lens choice, lighting, color palette and cinematic aesthetic. Keep it concise and scene-focused rather than overly technical."

MiniMax H3 Instead of sharing the prompts for each of these videos, I thought it would be more useful to share how I created that prompts. All of the videos were generated with text-to-video. First, find an image with the kind of scene, composition and mood you want to recreate. I used a few YouTube playlist thumbnails as references but Pinterest is also a great place to find inspiration. You can even use your own old or nostalgic photographs. Then upload the image to ChatGPT and ask it to describe the scene. The description it gives you can essentially become your text-to-video prompt. From there, you can generate completely new scenes with a similar composition, atmosphere and cinematic language. You can of course use the reference image directly with image-to-video or as a first frame. But if the original image isn't yours, I prefer using it only as visual inspiration and recreating the scene through text-to-video. This is the prompt I use with ChatGPT: "Describe the scene in this image in English, focusing primarily on what is happening, the characters, their actions and body language, the setting and the overall atmosphere. Also briefly describe the composition, framing, camera angle, approximate lens choice, lighting, color palette and cinematic aesthetic. Keep it concise and scene-focused rather than overly technical."

Kōda

51,449 просмотров • 12 дней назад

MiniMax H3 is now on Magnific and honestly, there’s a lot you can do with it. You can mix text, images, videos and audio in a single prompt up to 9 images, 3 videos and 3 audio references. Start creating now: It can generate up to 15s of 2K video with synced sound, including voice, music and effects. And you can go beyond generation too: edit clips, remove objects, transfer motion, and control the camera, character and voice. But the multi-reference workflow is probably my favorite. Give it your product, character, environment and motion references, and H3 pulls everything together. It feels like a much easier way to go from an idea to an actual finished video.

MiniMax H3 is now on Magnific and honestly, there’s a lot you can do with it. You can mix text, images, videos and audio in a single prompt up to 9 images, 3 videos and 3 audio references. Start creating now: It can generate up to 15s of 2K video with synced sound, including voice, music and effects. And you can go beyond generation too: edit clips, remove objects, transfer motion, and control the camera, character and voice. But the multi-reference workflow is probably my favorite. Give it your product, character, environment and motion references, and H3 pulls everything together. It feels like a much easier way to go from an idea to an actual finished video.

Kalsoom (ghotai )

47,888 просмотров • 7 дней назад

kling 2.6 motion control is actually insane for AI ads you can swap any person in a video with a different person and it looks completely realistic, and its way better than wan 2.2 this unlocks massive testing potential for ads you can test the same ad concept across multiple different ICPs instantly here's the workflow: >generate your new character with nano banana pro >use kling 2.6 motion control to replace the person in your existing video generate a new voice with >elevenlabs to match the character you can take one winning ad and instantly create 10 variations targeting: -different age groups -different genders -different ethnicities -different aesthetics this is the future of ad testing at scale

kling 2.6 motion control is actually insane for AI ads you can swap any person in a video with a different person and it looks completely realistic, and its way better than wan 2.2 this unlocks massive testing potential for ads you can test the same ad concept across multiple different ICPs instantly here's the workflow: >generate your new character with nano banana pro >use kling 2.6 motion control to replace the person in your existing video generate a new voice with >elevenlabs to match the character you can take one winning ad and instantly create 10 variations targeting: -different age groups -different genders -different ethnicities -different aesthetics this is the future of ad testing at scale

Miko

29,467 просмотров • 8 месяцев назад

✨ I updated Photo AI's video model to Kling AI 2.6 now By default it also generates audio (like Veo 3 does) and if you prompt for it it also produces voice, and it's pretty realistic too I'd say the cadence of the voice can be improved to be a bit more real, but it's definitely getting there It's just one click, just generate an AI photo first then hover over it and press [ 📼 Make video ] It's 3x more expensive, but I sell it below cost (5 credits) so that you all sign up to Photo AI and generate your UGC influencers with me 😊 Here's my 100% AI influencer telling what she thinks of the European Commission

✨ I updated Photo AI's video model to Kling AI 2.6 now By default it also generates audio (like Veo 3 does) and if you prompt for it it also produces voice, and it's pretty realistic too I'd say the cadence of the voice can be improved to be a bit more real, but it's definitely getting there It's just one click, just generate an AI photo first then hover over it and press [ 📼 Make video ] It's 3x more expensive, but I sell it below cost (5 credits) so that you all sign up to Photo AI and generate your UGC influencers with me 😊 Here's my 100% AI influencer telling what she thinks of the European Commission

@levelsio

323,674 просмотров • 8 месяцев назад

Seedance 2 just dropped. This was my first test and it changed everything. Perfect lip sync. Realistic movement. Multi-shot support. Natural voice completely human, zero robotic tone. And the SFX? It nailed the sound effects automatically. Every detail synced perfectly. This is a game-changer. From now on, automation isn't theory it's real. And yes, it's happening on Crafft. With a high-fidelity model like this + the right prompting rules and skills, you can generate endless variations that work every single time. Crafft agents will handle the whole process. One model, infinite outputs. You can try Seedance 2 on CapCut right now. RT + comment "PROMPT" and I'll send you the exact setup (follow so I can DM)

Seedance 2 just dropped. This was my first test and it changed everything. Perfect lip sync. Realistic movement. Multi-shot support. Natural voice completely human, zero robotic tone. And the SFX? It nailed the sound effects automatically. Every detail synced perfectly. This is a game-changer. From now on, automation isn't theory it's real. And yes, it's happening on Crafft. With a high-fidelity model like this + the right prompting rules and skills, you can generate endless variations that work every single time. Crafft agents will handle the whole process. One model, infinite outputs. You can try Seedance 2 on CapCut right now. RT + comment "PROMPT" and I'll send you the exact setup (follow so I can DM)

Ahmad

31,526 просмотров • 6 месяцев назад

NEW HTML VIDEO SKILL: Claude can now make photo-grid promo video ads like these. These are very popular offer-style ads that work well for bottom-of-funnel conversions. Brands will often run these when they're doing seasonal sales, for example. It's pure HTML, so it's very cheap to make because there's no video generation cost. You can create dozens of variants very fast and then test them in Meta. All you have to do is install the skills and give Claude your brand website. This skill is insane - it'll make the entire Ad and give you a link where you can download the mp4 file. > If you want more variants, just mention in the prompt > If you have a specific concept in mind, just mention in the prompt > If you want any edits on the generated ad, just mention in the prompt Comment Goose and I'll DM you the skill (must be following so I can DM you)

Shiv

92,070 просмотров • 1 месяц назад

Here's what you can also do with iDE Resurrector Origami: a PS2 Slim with an HDCombo-style case like back in the "golden days" of PS2 Slim modding 😊 Small specificity: the guy I made it for didn't want the flex cable to be visible, so a hole had to be made in the bottom shell, but the mod can also be done by discreetly running the flex through the outside 😉 Run perfectly with "cheap" 2Tb (and more) SSD The model of the bottom case can be found online (sorry I don't find back the link) oh! and I almost forgot, the switch allows you to disable the chip ;)

Here's what you can also do with iDE Resurrector Origami: a PS2 Slim with an HDCombo-style case like back in the "golden days" of PS2 Slim modding 😊 Small specificity: the guy I made it for didn't want the flex cable to be visible, so a hole had to be made in the bottom shell, but the mod can also be done by discreetly running the flex through the outside 😉 Run perfectly with "cheap" 2Tb (and more) SSD The model of the bottom case can be found online (sorry I don't find back the link) oh! and I almost forgot, the switch allows you to disable the chip ;)

Gusse

14,560 просмотров • 8 месяцев назад

LLM Artifacts Connected to Andrej Karpathy's LLM Knowledge base idea, I've been building out a fun way to generate dynamic artifacts from these knowledge bases with the goal of discovering and revealing meaningful and deeper insights. LLM KBs are hard to consume for humans, as I think they are more built for agents. So the question is, what form would be useful for humans to take actions and make important decisions? That's what I am trying to figure out with these artifacts. The artifact example shows a pulse on HN discussions around AI-related stories. The insights can go deeper, of course, but this is already super fun and thought-provoking, like some of my favorite podcasts. The format and depth matter a lot. The aggregation skills of agents are outstanding if you tune the prompts and skill carefully. I built this artifact generator in a few minutes through an agent skill, but I feel like there are so many ways that LLM-generated information can be used and consumed. Like generating deeper insights and analysis, and things that are just not feasible for humans today. The generated artifact (including its data and design) serves as reusable templates or can be updated in real-time via auomations, which is something I am also working on. It is truly an insane way to monitor and track information. Better than a newsletter. Better than newspapers. There is something about this that gets me really excited about the future of AI agents for knowledge generation and discovery. Lots of hidden gems everywhere just waiting to be discovered and acted on if the information is presented correctly. This is not perfect. The format, style/prose can be improved, but this is easy to customize via skill. You can personalize it to your liking. I feel like these dynamic artifacts are going to emerge as a strong new medium to stay on the cutting edge of things, both for agents and humans. My target is research, of course. This was just a basic example. Besides animation, I am also targeting other components like voice, videos, images, slides, etc. This space is full of opportunities to explore. Skill for this coming soon.

LLM Artifacts Connected to Andrej Karpathy's LLM Knowledge base idea, I've been building out a fun way to generate dynamic artifacts from these knowledge bases with the goal of discovering and revealing meaningful and deeper insights. LLM KBs are hard to consume for humans, as I think they are more built for agents. So the question is, what form would be useful for humans to take actions and make important decisions? That's what I am trying to figure out with these artifacts. The artifact example shows a pulse on HN discussions around AI-related stories. The insights can go deeper, of course, but this is already super fun and thought-provoking, like some of my favorite podcasts. The format and depth matter a lot. The aggregation skills of agents are outstanding if you tune the prompts and skill carefully. I built this artifact generator in a few minutes through an agent skill, but I feel like there are so many ways that LLM-generated information can be used and consumed. Like generating deeper insights and analysis, and things that are just not feasible for humans today. The generated artifact (including its data and design) serves as reusable templates or can be updated in real-time via auomations, which is something I am also working on. It is truly an insane way to monitor and track information. Better than a newsletter. Better than newspapers. There is something about this that gets me really excited about the future of AI agents for knowledge generation and discovery. Lots of hidden gems everywhere just waiting to be discovered and acted on if the information is presented correctly. This is not perfect. The format, style/prose can be improved, but this is easy to customize via skill. You can personalize it to your liking. I feel like these dynamic artifacts are going to emerge as a strong new medium to stay on the cutting edge of things, both for agents and humans. My target is research, of course. This was just a basic example. Besides animation, I am also targeting other components like voice, videos, images, slides, etc. This space is full of opportunities to explore. Skill for this coming soon.

elvis

31,295 просмотров • 4 месяцев назад

Most video tools can generate clips. Very few can maintain identity. That has been the real bottleneck in AI video creation. Kling O1 changes that. For the first time, creators can carry a character, style, and visual language across scenes without constant fixes. You can reference past clips, assets, or images and the output stays consistently on-model. No visual drift. No rework loops. No “this doesn’t look like the last shot” moments. It feels less like prompting a tool and more like working with a creative collaborator that remembers context. The impact is practical, not theoretical: → Faster production cycles → Lower iteration costs → Noticeably higher output quality This is what mature AI tooling looks like. Not louder features. Not bigger claims. Just reliability where it actually matters. Consistency is no longer the problem.

Most video tools can generate clips. Very few can maintain identity. That has been the real bottleneck in AI video creation. Kling O1 changes that. For the first time, creators can carry a character, style, and visual language across scenes without constant fixes. You can reference past clips, assets, or images and the output stays consistently on-model. No visual drift. No rework loops. No “this doesn’t look like the last shot” moments. It feels less like prompting a tool and more like working with a creative collaborator that remembers context. The impact is practical, not theoretical: → Faster production cycles → Lower iteration costs → Noticeably higher output quality This is what mature AI tooling looks like. Not louder features. Not bigger claims. Just reliability where it actually matters. Consistency is no longer the problem.

Darshal Jaitwar

141,038 просмотров • 7 месяцев назад

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

Jim Fan

594,014 просмотров • 2 лет назад

llama.cpp isn't just for text LLMs anymore. Pure C++ zero shot voice cloning just officially landed in mainline. Text generation was only step one. If you’re building autonomous local AI agents, real time voice assistants, or edge workflows, instant low latency audio is the missing piece. Thanks to PR #26254, Alibaba’s state of the art Qwen3 TTS model family is now natively supported directly inside the llama.cpp repository under the multimodal (mtmd) framework. No Python bloat. No massive PyTorch CUDA overhead. Just raw, hyper optimized C++ running GGUF voice weights. Here is why this native update is a massive deal for the open source local AI stack: # Multimodal Architecture (.gguf + mmproj) Qwen3-TTS splits the workload between the base language model backbone and a multimodal projection adapter. llama.cpp handles this using the llama-tts binary, mapping the text model alongside its --mmproj projector to process audio tokens seamlessly. # Zero Shot Voice Cloning in Seconds You don't need fine tuning or massive dataset training. Feed the C++ engine a single 5 to 10 second .wav audio sample using the --tts-speaker-file flag, and it accurately clones the exact timbre, tone, and accent on the fly. # Real World T4 GPU Benchmark & Resource FootprintRunning the 1.7B Base model in 8-bit quantization (Q8_0): - VRAM Footprint: ~7 GB peak VRAM during active zero-shot cloning. - Audio Quality: Studio grade, natural-sounding voice output in seconds. • - Execution: Direct execution via native compiled binaries or sub process calls. # Coming Next to llama-server (PR #26603) Beyond CLI execution, a native POST /tts HTTP endpoint is currently being added to llama-server, which will soon allow you to trigger voice generation directly via standard REST API requests! # quick note on Colab compilation: Because this code was merged into mainline very recently, pre-built third-party binaries haven't fully caught up yet. Compiling llama-tts directly from source on Google Colab's free CPU instance can take about 1 hour (or ~1-2 minutes if targeting single GPU arch like -DCMAKE_CUDA_ARCHITECTURES=75). Be patient during the build step, or compile it locally on your own rig for instant execution! To test this out yourself, I built a zero config Google Colab notebook that compiles llama.cpp, downloads the Q8_0 GGUF files from HuggingFace, and spins up an interactive Gradio Studio UI so you can record/upload 3 second clips and clone voices in real time. Stop sleeping on native C++ audio. The era of bulky Python audio pipelines is officially over. Links to the free Google Colab notebook and the official ggml org GGUF HuggingFace model repository are in the replies below! available in q4 and q8 both variants, 1 GB and 1.85 GBs respectively (requires additional ~500MB mmproj gguf) Are you building local voice agents yet? What does your current audio stack look like? Drop your setups below!

llama.cpp isn't just for text LLMs anymore. Pure C++ zero shot voice cloning just officially landed in mainline. Text generation was only step one. If you’re building autonomous local AI agents, real time voice assistants, or edge workflows, instant low latency audio is the missing piece. Thanks to PR #26254, Alibaba’s state of the art Qwen3 TTS model family is now natively supported directly inside the llama.cpp repository under the multimodal (mtmd) framework. No Python bloat. No massive PyTorch CUDA overhead. Just raw, hyper optimized C++ running GGUF voice weights. Here is why this native update is a massive deal for the open source local AI stack: # Multimodal Architecture (.gguf + mmproj) Qwen3-TTS splits the workload between the base language model backbone and a multimodal projection adapter. llama.cpp handles this using the llama-tts binary, mapping the text model alongside its --mmproj projector to process audio tokens seamlessly. # Zero Shot Voice Cloning in Seconds You don't need fine tuning or massive dataset training. Feed the C++ engine a single 5 to 10 second .wav audio sample using the --tts-speaker-file flag, and it accurately clones the exact timbre, tone, and accent on the fly. # Real World T4 GPU Benchmark & Resource FootprintRunning the 1.7B Base model in 8-bit quantization (Q8_0): - VRAM Footprint: ~7 GB peak VRAM during active zero-shot cloning. - Audio Quality: Studio grade, natural-sounding voice output in seconds. • - Execution: Direct execution via native compiled binaries or sub process calls. # Coming Next to llama-server (PR #26603) Beyond CLI execution, a native POST /tts HTTP endpoint is currently being added to llama-server, which will soon allow you to trigger voice generation directly via standard REST API requests! # quick note on Colab compilation: Because this code was merged into mainline very recently, pre-built third-party binaries haven't fully caught up yet. Compiling llama-tts directly from source on Google Colab's free CPU instance can take about 1 hour (or ~1-2 minutes if targeting single GPU arch like -DCMAKE_CUDA_ARCHITECTURES=75). Be patient during the build step, or compile it locally on your own rig for instant execution! To test this out yourself, I built a zero config Google Colab notebook that compiles llama.cpp, downloads the Q8_0 GGUF files from HuggingFace, and spins up an interactive Gradio Studio UI so you can record/upload 3 second clips and clone voices in real time. Stop sleeping on native C++ audio. The era of bulky Python audio pipelines is officially over. Links to the free Google Colab notebook and the official ggml org GGUF HuggingFace model repository are in the replies below! available in q4 and q8 both variants, 1 GB and 1.85 GBs respectively (requires additional ~500MB mmproj gguf) Are you building local voice agents yet? What does your current audio stack look like? Drop your setups below!

Alok

47,881 просмотров • 22 дней назад