Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports...

LiorOnAI's profile picture

Lior Alexander

116,186 subscribers

61,122 Aufrufe • vor 6 Monaten •via X (Twitter)

BildungNachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Microsoft did it again! Speech AI models have a major limitation. They slice long recordings into tiny chunks, lose track of who's speaking, and forget all context halfway through. This is exactly what Microsoft's VibeVoice solves. It's an open-source family of frontier voice AI models for both speech recognition and speech generation. Here's what it can do: > VibeVoice-ASR processes up to 60 minutes of audio in a single pass. No chunking. It outputs structured transcriptions with who spoke, when they spoke, and what they said. > You can feed it custom hotwords like names, technical jargon, or domain-specific terms. The model uses them to significantly improve accuracy on specialized content. > VibeVoice-TTS generates up to 90 minutes of multi-speaker speech with up to 4 distinct speakers. Natural turn-taking, emotional expression, all in one pass. > VibeVoice-Realtime is a 0.5B streaming TTS model with ~300ms first-audio latency. Small enough to deploy practically anywhere. All of this is powered by continuous speech tokenizers running at just 7.5 Hz. This ultra-low frame rate preserves audio quality while making long sequences computationally feasible. I have shared the link to the GitHub repo in the replies!
1:10

Microsoft did it again! Speech AI models have a major limitation. They slice long recordings into tiny chunks, lose track of who's speaking, and forget all context halfway through. This is exactly what Microsoft's VibeVoice solves. It's an open-source family of frontier voice AI models for both speech recognition and speech generation. Here's what it can do: > VibeVoice-ASR processes up to 60 minutes of audio in a single pass. No chunking. It outputs structured transcriptions with who spoke, when they spoke, and what they said. > You can feed it custom hotwords like names, technical jargon, or domain-specific terms. The model uses them to significantly improve accuracy on specialized content. > VibeVoice-TTS generates up to 90 minutes of multi-speaker speech with up to 4 distinct speakers. Natural turn-taking, emotional expression, all in one pass. > VibeVoice-Realtime is a 0.5B streaming TTS model with ~300ms first-audio latency. Small enough to deploy practically anywhere. All of this is powered by continuous speech tokenizers running at just 7.5 Hz. This ultra-low frame rate preserves audio quality while making long sequences computationally feasible. I have shared the link to the GitHub repo in the replies!

Akshay 🚀

45,256 Aufrufe • vor 4 Monaten

🚨 JUST IN: MICROSOFT just open sourced a VOICE AI THAT TRANSCRIBES 60 MINUTES OF AUDIO in a single pass. 100% FREE. It knows who spoke. It knows when they spoke. It knows exactly what they said. All in one shot. No chunking. No context loss. It's called VibeVoice. Not a transcription tool. Not a basic speech to text wrapper. A frontier voice AI family with ASR, TTS, and real time streaming. All open source. All free. Here's what it actually does 👇 VibeVoice ASR - Speech Recognition: → Processes 60 minutes of continuous audio in a single pass → Never slices audio into chunks so global context is never lost → Identifies WHO spoke, WHEN they spoke and WHAT they said simultaneously → Supports customized hotwords for domain specific accuracy → Works in 50+ languages natively → Already adopted by Hugging Face Transformers library → Already being built on by the open source community BY PEOPLE WHO HAD NO IDEA THIS LEVEL OF ACCURACY WAS ALREADY FREE. VibeVoice TTS - Text to Speech: → Generates up to 90 minutes of speech in a single pass → Supports up to 4 distinct speakers in one conversation → Natural turn taking and speaker consistency throughout → Expressive speech that captures emotional nuances → Supports English, Chinese and multiple other languages VibeVoice Realtime - Streaming TTS: → Only 300 millisecond first audible latency → Streams text input in real time → 0.5B parameters so it actually deploys anywhere → Robust long form generation up to 10 minutes → Lightweight enough for production use today The core innovation nobody is talking about: Most voice AI models slice long audio into short chunks. Every time they slice, they lose context. Speaker tracking breaks. Semantic coherence breaks. Accuracy drops. VibeVoice uses continuous speech tokenizers running at an ultra low frame rate of 7.5 Hz. This preserves audio fidelity while dramatically boosting computational efficiency. The entire 60 minutes stays in context. Nothing gets lost. Nobody gets misidentified. The numbers: → VibeVoice ASR 7B - available now on Hugging Face → VibeVoice Realtime 0.5B - try it on Colab right now → 50+ supported languages → 11 distinct English voice styles → 9 multilingual speaker voices → Already integrated into Hugging Face Transformers → Finetuning code now available The wildest part? A voice powered input method called Vibing just built itself on top of VibeVoice ASR. Available on macOS and Windows right now. The open source community is already shipping products on top of this. 100% Open Source. Free to use. Free to fine tune. Free to build on. 🔖 Save this before your competitors find it first. 👇
0:44

🚨 JUST IN: MICROSOFT just open sourced a VOICE AI THAT TRANSCRIBES 60 MINUTES OF AUDIO in a single pass. 100% FREE. It knows who spoke. It knows when they spoke. It knows exactly what they said. All in one shot. No chunking. No context loss. It's called VibeVoice. Not a transcription tool. Not a basic speech to text wrapper. A frontier voice AI family with ASR, TTS, and real time streaming. All open source. All free. Here's what it actually does 👇 VibeVoice ASR - Speech Recognition: → Processes 60 minutes of continuous audio in a single pass → Never slices audio into chunks so global context is never lost → Identifies WHO spoke, WHEN they spoke and WHAT they said simultaneously → Supports customized hotwords for domain specific accuracy → Works in 50+ languages natively → Already adopted by Hugging Face Transformers library → Already being built on by the open source community BY PEOPLE WHO HAD NO IDEA THIS LEVEL OF ACCURACY WAS ALREADY FREE. VibeVoice TTS - Text to Speech: → Generates up to 90 minutes of speech in a single pass → Supports up to 4 distinct speakers in one conversation → Natural turn taking and speaker consistency throughout → Expressive speech that captures emotional nuances → Supports English, Chinese and multiple other languages VibeVoice Realtime - Streaming TTS: → Only 300 millisecond first audible latency → Streams text input in real time → 0.5B parameters so it actually deploys anywhere → Robust long form generation up to 10 minutes → Lightweight enough for production use today The core innovation nobody is talking about: Most voice AI models slice long audio into short chunks. Every time they slice, they lose context. Speaker tracking breaks. Semantic coherence breaks. Accuracy drops. VibeVoice uses continuous speech tokenizers running at an ultra low frame rate of 7.5 Hz. This preserves audio fidelity while dramatically boosting computational efficiency. The entire 60 minutes stays in context. Nothing gets lost. Nobody gets misidentified. The numbers: → VibeVoice ASR 7B - available now on Hugging Face → VibeVoice Realtime 0.5B - try it on Colab right now → 50+ supported languages → 11 distinct English voice styles → 9 multilingual speaker voices → Already integrated into Hugging Face Transformers → Finetuning code now available The wildest part? A voice powered input method called Vibing just built itself on top of VibeVoice ASR. Available on macOS and Windows right now. The open source community is already shipping products on top of this. 100% Open Source. Free to use. Free to fine tune. Free to build on. 🔖 Save this before your competitors find it first. 👇

Kanika

221,026 Aufrufe • vor 4 Monaten

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓
0:41

Text-to-speech is moving way too fast. Just a few days ago, I tweeted about PersonaPlex-7B, NVIDIA's new open source TTS ( And today, Qwen just open-sourced Qwen3-TTS 🤯 It’s a revolutionary text-to-speech model built for control. Not just about generating speech, but about shaping how it sounds directly from language. You can guide the pace, the tone, and the expressiveness straight from text, without touching audio graphs or hand-tuning parameters. That’s the real shift! What makes Qwen3-TTS stand out is how practical it already is: → voice cloning from just a few seconds of audio → voice creation without any reference sample → support for 10 languages out of the box → end-to-end latency down to ~97ms → works in both streaming and non-streaming setups The models come in two sizes (0.6B and 1.7B), so you can trade off quality and hardware cost depending on your setup. You can work with curated voices, designed voices, or cloned ones, and it integrates cleanly with vLLM for production use. It also ships as a simple Python package you can pip install. If you’re building real-time voice systems, this removes a lot of friction! 100% free and open source. I put the repo in the 🧵↓

Charly Wargnier

59,144 Aufrufe • vor 6 Monaten

This is the fastest text-to-speech and speech-to-text API out there. It's under 250 ms to first-byte latency. That's what we need to deploy conversational AI applications everywhere! (I can't wait for the death of automated voice machines.) Look at the attached video. For the first time, I could have a conversation with an AI agent in real time without awkward pauses. This technology is Aura from Deepgram. It's the fastest text-to-speech option I've seen. Here is a GitHub repository with the demo in the video: You can use it to build a conversational AI application using natural voice. You'll need a Deepgram API Key and an OpenAI API Key. Thanks to Deepgram for giving me access to Aura and collaborating with me on this post. If you want to learn more about Aura, here is the announcement post:
3:38

This is the fastest text-to-speech and speech-to-text API out there. It's under 250 ms to first-byte latency. That's what we need to deploy conversational AI applications everywhere! (I can't wait for the death of automated voice machines.) Look at the attached video. For the first time, I could have a conversation with an AI agent in real time without awkward pauses. This technology is Aura from Deepgram. It's the fastest text-to-speech option I've seen. Here is a GitHub repository with the demo in the video: You can use it to build a conversational AI application using natural voice. You'll need a Deepgram API Key and an OpenAI API Key. Thanks to Deepgram for giving me access to Aura and collaborating with me on this post. If you want to learn more about Aura, here is the announcement post:

Santiago

185,640 Aufrufe • vor 2 Jahren

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.
1:21

Mistral just released Voxtral TTS, an open-weight voice model that tries to match ElevenLabs while keeping it free. A 3-billionn param model that fits on a laptop and runs 6X faster than real-time speech. It is split into 3 parts that handle different jobs. - The 3.4B part is the main language brain that understands the text and plans what should be spoken. - The 390M part turns that plan into speech features like rhythm, pronunciation, and tone. - The 300M part converts those speech features into the final compressed audio signal you can play. That split helps because each part only does the job it is best at, which can make the whole system faster and lighter than one giant model doing everything alone. Mistral claims about 90ms to first audio, about 6x faster than real-time speech output, about 3GB RAM after quantization, 9 languages, and voice copying from 5s of audio, even across languages. Its internal tests report 62.8% preference over ElevenLabs Flash on default voices and 69.9% on custom voices, but the bigger point is that companies can keep speech data on their own machines. IMO, enterprise buyers usually care most about control, cost, and latency, So this model should give it great push for voice-based AI apps/products.

Rohan Paul

15,672 Aufrufe • vor 4 Monaten

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥
0:42

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Charly Wargnier

565,502 Aufrufe • vor 6 Monaten

Listen, speak, handle interruptions, and call tools in one live conversation. 🎙️ Introducing NVIDIA-NemotronLabs-VoiceChat-11B, NVIDIA’s end-to-end full-duplex model for real-time voice agents. 🤖 🏆 It ranks #2 among open full-duplex models on both VoiceBench and Full-Duplex-Bench 1.0. ⚡ Natural turn-taking responds in ~448 ms, while barge-in lets users interrupt the model with ~480 ms latency. 🛠️ The first open full-duplex model with live tool calling. One unified system handles speech understanding, generation, transcription, and tool execution while keeping the conversation flowing. 🧠 Built with a Fast Conformer, Nemotron Nano v2, and a TTS decoder, trained on ~550K hours of real and synthetic speech. 📜 Research use only. OpenMDW 1.1.
1:24

Listen, speak, handle interruptions, and call tools in one live conversation. 🎙️ Introducing NVIDIA-NemotronLabs-VoiceChat-11B, NVIDIA’s end-to-end full-duplex model for real-time voice agents. 🤖 🏆 It ranks #2 among open full-duplex models on both VoiceBench and Full-Duplex-Bench 1.0. ⚡ Natural turn-taking responds in ~448 ms, while barge-in lets users interrupt the model with ~480 ms latency. 🛠️ The first open full-duplex model with live tool calling. One unified system handles speech understanding, generation, transcription, and tool execution while keeping the conversation flowing. 🧠 Built with a Fast Conformer, Nemotron Nano v2, and a TTS decoder, trained on ~550K hours of real and synthetic speech. 📜 Research use only. OpenMDW 1.1.

ModelScope

21,832 Aufrufe • vor 9 Tagen

🇨🇳 Another great Chinese Model, OmniHuman-1.5 from ByteDance Turns 1 image plus a voice track into expressive avatar video by pairing a System 1 and System 2 inspired planner with a Diffusion Transformer, Produces coherent motion for over 1 minute with moving camera and multi character scenes. Most avatar models move to the beat of the audio but miss meaning, so gestures feel generic and emotions feel shallow. The fix here is a Multimodal LLM planner that listens to the speech and drafts a structured plan describing intent, emotions, beats, and high level actions, which gives the motion engine clear semantic targets instead of only rhythm. The motion engine is a Multimodal Diffusion Transformer that fuses the plan with audio, the single reference image, and optional text prompts, then synthesizes continuous body, face, and head motion that matches both words and tone. A key trick is a Pseudo Last Frame, a synthetic target that summarizes the next expected state, which stabilizes fusion across modalities and keeps motion consistent over long spans. From just 1 image and speech, the system outputs speaking avatars with synchronized lips, context aware gestures, and continuous camera movement, and it also supports multi character interactions without manual choreography. Reported results show strong lip sync accuracy, high video quality, natural motion, and close match to text prompts, and the same setup works on nonhuman characters too.
0:18

🇨🇳 Another great Chinese Model, OmniHuman-1.5 from ByteDance Turns 1 image plus a voice track into expressive avatar video by pairing a System 1 and System 2 inspired planner with a Diffusion Transformer, Produces coherent motion for over 1 minute with moving camera and multi character scenes. Most avatar models move to the beat of the audio but miss meaning, so gestures feel generic and emotions feel shallow. The fix here is a Multimodal LLM planner that listens to the speech and drafts a structured plan describing intent, emotions, beats, and high level actions, which gives the motion engine clear semantic targets instead of only rhythm. The motion engine is a Multimodal Diffusion Transformer that fuses the plan with audio, the single reference image, and optional text prompts, then synthesizes continuous body, face, and head motion that matches both words and tone. A key trick is a Pseudo Last Frame, a synthetic target that summarizes the next expected state, which stabilizes fusion across modalities and keeps motion consistent over long spans. From just 1 image and speech, the system outputs speaking avatars with synchronized lips, context aware gestures, and continuous camera movement, and it also supports multi character interactions without manual choreography. Reported results show strong lip sync accuracy, high video quality, natural motion, and close match to text prompts, and the same setup works on nonhuman characters too.

Rohan Paul

63,859 Aufrufe • vor 11 Monaten

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer
3:03

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Rohan Paul

96,354 Aufrufe • vor 1 Jahr

Introducing Lightning V3 - it beats every model we tested against. ElevenLabs, Cartesia, OpenAI. Lightning sets a new SOTA with V3 in conversational text-to-speech. → Highest MOS score for conversational TTS at 3.9 → ~76% win rate vs gpt-4o-mini-tts on naturalness → 15 languages with mid-sentence code-switching → Built from scratch for voice agents, not read-aloud Every TTS model sounds clean in a demo. You type a sentence and you get beautiful audio. Voice agents don't work that way. They stream. They're generating audio in real-time chunks with half the context missing. That's where everything breaks. A great reading voice and a great conversational voice are fundamentally different things. A conversational voice has to sound like it's thinking - with the pauses, the rhythm shifts, the reactions. It has to handle the way real people actually talk, including switching languages mid-sentence. That's what V3 does. V3.1 also ships voice cloning. 5 to 15 seconds of audio, no fine-tuning, production-grade clone across 15 languages. Blog link in the comments.
1:33

Introducing Lightning V3 - it beats every model we tested against. ElevenLabs, Cartesia, OpenAI. Lightning sets a new SOTA with V3 in conversational text-to-speech. → Highest MOS score for conversational TTS at 3.9 → ~76% win rate vs gpt-4o-mini-tts on naturalness → 15 languages with mid-sentence code-switching → Built from scratch for voice agents, not read-aloud Every TTS model sounds clean in a demo. You type a sentence and you get beautiful audio. Voice agents don't work that way. They stream. They're generating audio in real-time chunks with half the context missing. That's where everything breaks. A great reading voice and a great conversational voice are fundamentally different things. A conversational voice has to sound like it's thinking - with the pauses, the rhythm shifts, the reactions. It has to handle the way real people actually talk, including switching languages mid-sentence. That's what V3 does. V3.1 also ships voice cloning. 5 to 15 seconds of audio, no fine-tuning, production-grade clone across 15 languages. Blog link in the comments.

Sudarshan Kamath

71,298 Aufrufe • vor 4 Monaten

Someone open-sourced a 66M parameter model that beats ElevenLabs, OpenAI, and Gemini and runs entirely offline on a Raspberry Pi. it's called Supertonic, a text-to-speech model so fast it turns an entire webpage into audio in under 1 second. locally. offline. for free. text-to-speech has lived in the cloud for years. every spoken character was an API call and a fraction of a cent. Supertonic 3 kills that entirely. the model is 99M parameters. ships as a single ONNX file. hits 167× faster than real-time on a laptop CPU. that's ~1,263 characters of speech per second. other open systems sit at 55–287. no cloud. no API. no GPU. → 31 languages, no separate adapters → works inside a browser tab (WebGPU/WASM) → handles phone numbers, currency, dates — no preprocessing → inline tags for laugh and breath → Turns an entire webpage into audio in under 1 second → 44.1kHz studio-grade WAV, no upsampler needed` and on real-world text, "$5.2M", "(212) 555-0142 ext. 402", "30kph”, it's the only one that reads them correctly. ElevenLabs Flash, OpenAI TTS-1, Gemini 2.5 Flash all fail. Supertonic passes. 11.2k stars. 100% Open Source.
1:00

Someone open-sourced a 66M parameter model that beats ElevenLabs, OpenAI, and Gemini and runs entirely offline on a Raspberry Pi. it's called Supertonic, a text-to-speech model so fast it turns an entire webpage into audio in under 1 second. locally. offline. for free. text-to-speech has lived in the cloud for years. every spoken character was an API call and a fraction of a cent. Supertonic 3 kills that entirely. the model is 99M parameters. ships as a single ONNX file. hits 167× faster than real-time on a laptop CPU. that's ~1,263 characters of speech per second. other open systems sit at 55–287. no cloud. no API. no GPU. → 31 languages, no separate adapters → works inside a browser tab (WebGPU/WASM) → handles phone numbers, currency, dates — no preprocessing → inline tags for laugh and breath → Turns an entire webpage into audio in under 1 second → 44.1kHz studio-grade WAV, no upsampler needed` and on real-world text, "$5.2M", "(212) 555-0142 ext. 402", "30kph”, it's the only one that reads them correctly. ElevenLabs Flash, OpenAI TTS-1, Gemini 2.5 Flash all fail. Supertonic passes. 11.2k stars. 100% Open Source.

Superman

30,261 Aufrufe • vor 26 Tagen

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:
1:39

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

Tencent Hy

89,257 Aufrufe • vor 11 Monaten

I just added real-time AI translation into Chormex using GPT-Realtime-2… and this feels absolutely surreal. It works across YouTube videos, live streams, meetings, presentations, basically anywhere audio is playing inside Chrome. You can watch translated speech in real time while simultaneously using Codex on top of the live context. “Summarize this.” “What are the key points?” “Turn this into notes.” “Explain what they mean.” “Organize the discussion.” …all while the video or meeting is still happening. It genuinely feels like browsers are evolving into real-time AI operating systems. We are getting dangerously close to a world where language barriers on the internet completely disappear.
1:29

I just added real-time AI translation into Chormex using GPT-Realtime-2… and this feels absolutely surreal. It works across YouTube videos, live streams, meetings, presentations, basically anywhere audio is playing inside Chrome. You can watch translated speech in real time while simultaneously using Codex on top of the live context. “Summarize this.” “What are the key points?” “Turn this into notes.” “Explain what they mean.” “Organize the discussion.” …all while the video or meeting is still happening. It genuinely feels like browsers are evolving into real-time AI operating systems. We are getting dangerously close to a world where language barriers on the internet completely disappear.

CHOI

191,487 Aufrufe • vor 3 Monaten

the real UGC industry isn't dying, it already died created this AI UGC in just a few minutes using my brand new system, from first prompt to final video > product stays in frame exactly where it should > few different people in same one video > can be adapted to any script, character, and environment > voice sounds like a real person, not a robot reading a script > movements are smooth and controlable in every shot > no time limit - make it as long as you need with this level of AI UGC, you can create and test 50x more ads than with real creators while saving $300-$500+ per video
0:49

the real UGC industry isn't dying, it already died created this AI UGC in just a few minutes using my brand new system, from first prompt to final video > product stays in frame exactly where it should > few different people in same one video > can be adapted to any script, character, and environment > voice sounds like a real person, not a robot reading a script > movements are smooth and controlable in every shot > no time limit - make it as long as you need with this level of AI UGC, you can create and test 50x more ads than with real creators while saving $300-$500+ per video

MAX

18,229 Aufrufe • vor 4 Monaten

I just built an open NotebookLM clone! Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙️ The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works, by building it step-by-step! So by the end of this video, you'll learn how to: ↳ Process multimodal data (text, audio, video, URLs, and YouTube videos) into a format ready for LLMs ↳ Store everything in a vector database for fast retrieval ↳ Add a memory layer that remembers conversations and preferences for a personalized experience ↳ Chat with your knowledge base or generate podcasts using a fully open-source, locally running text-to-speech model The podcast generation feature is my favorite part! There's something powerful about turning written content into conversational audio that you can listen to while doing something else. The entire code is 100% open-source. I've shared a link in the replies! ____ Don't forget to drop a like if you enjoy my videos. It shows me I should be making more content like this. Cheers! :)
20:04

I just built an open NotebookLM clone! Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙️ The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works, by building it step-by-step! So by the end of this video, you'll learn how to: ↳ Process multimodal data (text, audio, video, URLs, and YouTube videos) into a format ready for LLMs ↳ Store everything in a vector database for fast retrieval ↳ Add a memory layer that remembers conversations and preferences for a personalized experience ↳ Chat with your knowledge base or generate podcasts using a fully open-source, locally running text-to-speech model The podcast generation feature is my favorite part! There's something powerful about turning written content into conversational audio that you can listen to while doing something else. The entire code is 100% open-source. I've shared a link in the replies! ____ Don't forget to drop a like if you enjoy my videos. It shows me I should be making more content like this. Cheers! :)

Akshay 🚀

83,187 Aufrufe • vor 10 Monaten

We believe the path to general intelligence demands models that see, hear, and act in the world around us. Text alone can’t capture the crackle in a colleague’s voice or a sarcasm in a remark, the warning beep of machinery, or the ambient clues that define context. Kestrel is our new on-device speech-to-speech audio reasoning model: 1.1B parameters, 25 tokens/sec, 0.38 s TTF, F1-leader among lightweight audio LLMs. It processes raw audio in real time, infers intent, emotion, and environment, and responds - all without an internet connection. With Kestrel, agents gain ears: they can negotiate in meetings, monitor critical systems, or simply understand the room. It’s the next milestone toward securely embedding AGI everywhere. We’re excited to share early demos and welcome collaborators as we approach launch. (Shown here: Kestrel’s early version on our on-device demo app).
0:37

We believe the path to general intelligence demands models that see, hear, and act in the world around us. Text alone can’t capture the crackle in a colleague’s voice or a sarcasm in a remark, the warning beep of machinery, or the ambient clues that define context. Kestrel is our new on-device speech-to-speech audio reasoning model: 1.1B parameters, 25 tokens/sec, 0.38 s TTF, F1-leader among lightweight audio LLMs. It processes raw audio in real time, infers intent, emotion, and environment, and responds - all without an internet connection. With Kestrel, agents gain ears: they can negotiate in meetings, monitor critical systems, or simply understand the room. It’s the next milestone toward securely embedding AGI everywhere. We’re excited to share early demos and welcome collaborators as we approach launch. (Shown here: Kestrel’s early version on our on-device demo app).

Decompute AI

68,196 Aufrufe • vor 1 Jahr

Mira Murati says the next AI interface is not a chatbot. It is a model that can listen, see, interrupt, and respond in real time. "The models that we work with today, they're very turn-based." "You talk, they talk, then they go off and think." "While they're thinking, it's almost like they're deaf and blind. They cannot perceive anything else about what's going on." Her new frame: interaction models. "They're not turn-based. They're more like time-based interaction." "They're continuously taking in audio, text, video, and continuously providing output." "We cut this up in chunks of 20 milliseconds... this enables you to catch things like interruptions and simultaneous speech." That is the product shift: from AI as a prompt box to AI as a high-bandwidth collaborator.
1:33

Mira Murati says the next AI interface is not a chatbot. It is a model that can listen, see, interrupt, and respond in real time. "The models that we work with today, they're very turn-based." "You talk, they talk, then they go off and think." "While they're thinking, it's almost like they're deaf and blind. They cannot perceive anything else about what's going on." Her new frame: interaction models. "They're not turn-based. They're more like time-based interaction." "They're continuously taking in audio, text, video, and continuously providing output." "We cut this up in chunks of 20 milliseconds... this enables you to catch things like interruptions and simultaneous speech." That is the product shift: from AI as a prompt box to AI as a high-bandwidth collaborator.

Karl Mehta

191,606 Aufrufe • vor 1 Monat

training a model that takes a text prompt and generates audio that renders video on an oscilloscope AgenC agents live inside worlds the model generates the pipeline: real videos -> edge detection -> vectorization -> path ordering -> 192kHz 3-channel WAV where X/Y control beam position and Z controls beam intensity 3 values per timestep. that's all the model is learning. compare that to video gen models trying to predict millions of pixels per frame. transformers are already great at sequence prediction and that's literally all this is. waveform generation the output IS the playback. generate the audio, feed it to a scope, it draws the scene in real-time. there's no rendering step. it's analog so there's no pixel grid. you get continuous curves and effectively infinite resolution bootstrapped with procedural data, lissajous curves, wireframe 3D, stick figures, then scaled on real-world video converted to trace format. 90 TB of source video the model learns edges, contours, spatial relationships, motion. once it has that, describing a scene it's never seen is novel trajectory through the same learned space. generative geometry
0:22

training a model that takes a text prompt and generates audio that renders video on an oscilloscope AgenC agents live inside worlds the model generates the pipeline: real videos -> edge detection -> vectorization -> path ordering -> 192kHz 3-channel WAV where X/Y control beam position and Z controls beam intensity 3 values per timestep. that's all the model is learning. compare that to video gen models trying to predict millions of pixels per frame. transformers are already great at sequence prediction and that's literally all this is. waveform generation the output IS the playback. generate the audio, feed it to a scope, it draws the scene in real-time. there's no rendering step. it's analog so there's no pixel grid. you get continuous curves and effectively infinite resolution bootstrapped with procedural data, lissajous curves, wireframe 3D, stick figures, then scaled on real-world video converted to trace format. 90 TB of source video the model learns edges, contours, spatial relationships, motion. once it has that, describing a scene it's never seen is novel trajectory through the same learned space. generative geometry

tetsuo

18,397 Aufrufe • vor 5 Monaten

🚀🚀🚀Introducing HY World 1.5 (WorldPlay)! We have now open-sourced the most systemized, comprehensive real-time world model framework in the industry. In HY World 1.5, we develop WorldPlay, a streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency, resolving the trade-off between speed and memory that limits current methods. You can generate and explore 3D worlds simply by inputting text or images. Walk, look around, and interact like you're playing a game. Highlights: 🔹Real-Time: Generates long-horizon streaming video at 24 FPS with superior consistency. 🔹Geometric Consistency: Achieved using a Reconstituted Context Memory mechanism to dynamically rebuild context from past frames to alleviate memory attenuation 🔹Robust Control: Uses a Dual Action Representation for robust response to user keyboard and mouse inputs. 🔹Versatile Applications: Supports both first-person and third-person perspectives, enabling applications like promptable events and infinite world extension. 👉🏻Try it now: 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical Report:
1:06

🚀🚀🚀Introducing HY World 1.5 (WorldPlay)! We have now open-sourced the most systemized, comprehensive real-time world model framework in the industry. In HY World 1.5, we develop WorldPlay, a streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency, resolving the trade-off between speed and memory that limits current methods. You can generate and explore 3D worlds simply by inputting text or images. Walk, look around, and interact like you're playing a game. Highlights: 🔹Real-Time: Generates long-horizon streaming video at 24 FPS with superior consistency. 🔹Geometric Consistency: Achieved using a Reconstituted Context Memory mechanism to dynamically rebuild context from past frames to alleviate memory attenuation 🔹Robust Control: Uses a Dual Action Representation for robust response to user keyboard and mouse inputs. 🔹Versatile Applications: Supports both first-person and third-person perspectives, enabling applications like promptable events and infinite world extension. 👉🏻Try it now: 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical Report:

Tencent Hy

189,802 Aufrufe • vor 7 Monaten