Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Listen, speak, handle interruptions, and call tools in one live conversation. 🎙️ Introducing NVIDIA-NemotronLabs-VoiceChat-11B, NVIDIA’s end-to-end full-duplex model for real-time voice agents. 🤖 🏆 It ranks #2 among open full-duplex models on both VoiceBench and Full-Duplex-Bench 1.0. ⚡ Natural turn-taking responds in ~448 ms, while barge-in lets users interrupt... show more

ModelScope

15,721 subscribers

22,424 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 Kommentare

Profilbild von Alice The Ai Expert

Alice The Ai Expertvor 1 Monat

Real time voice that actually listens. 448ms responses, live tool calls, zero awkward pauses.

Profilbild von Mahimai Raja J ‎

Mahimai Raja J ‎vor 1 Monat

Everything good but the license bites 😃

Profilbild von Polash Khan

Polash Khanvor 1 Monat

Wow, that's super impressive! Sounds like a game-changer for voice agents. Congrats on the ranking!

Profilbild von Akmal Sultanov

Akmal Sultanovvor 1 Monat

it has pretty high latency for a duplex model for some reason. it doesn't sound like 500ms response latency, more like >1.5-2s latency. awkward...

Profilbild von Karan Singla

Karan Singlavor 1 Monat

Full-duplex fixes the speed. The catch is the glass box: a cascade gives you a clean per-utterance record you can audit, duplex blurs who said what and when. Getting duplex latency without losing that structure is the actual hard part.

Profilbild von RAZA | AI EXPLORER

RAZA | AI EXPLORERvor 1 Monat

NVIDIA's new end-to-end voice model brings seamless conversations and live tool use together.

Profilbild von AI Mastery Guide

AI Mastery Guidevor 1 Monat

Barge-in latency that low finally makes voice agents feel real.

Ähnliche Videos

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.

🚨NEW: CHATGPT LIVE VOICE MODEL NOW DELIVERS REAL-TIME CONVERSATIONAL TRANSLATION Built on a full-duplex architecture, GPT-Live is a new generation of voice models that can listen and speak simultaneously, enabling more natural human-AI interactions. The breakthrough unlocks fluid back-and-forth conversations, real-time translation, and a far more human sense of timing and context.

Coin Bureau

29,085 Aufrufe • vor 2 Monaten

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

NVIDIA just removed the biggest friction point in Voice AI 🤯 They've open-sourced PersonaPlex-7B, a full-duplex conversational model that can listen and speak at the same time. Instead of waiting for you to finish talking, it uses a dual-stream architecture to process and respond in real-time. 100% Open-Source and Free.

Simplifying AI

278,970 Aufrufe • vor 8 Monaten

🥳 Introducing MiniCPM-o 4.5 The first full-duplex omni-modal LLM in open-source community 🎬🎙️ 🔥 Key Highlights: • Full-duplex Omni-modal Live Streaming: The model can see, listen, and speak simultaneously in a real-time conversation without mutual blocking • Proactive Interaction: Moving beyond reactive QA to performing proactive interaction, such as initiating reminders • Leading Performance: Scoring 77.6 on OpenCompass, it outperforms GPT-4o & Gemini 2.0 Pro in vision-language tasks with 9B params The best part? You can experience all above on your PC! #MiniCPM #OpenSource #MultimodalAI #LLM

🥳 Introducing MiniCPM-o 4.5 The first full-duplex omni-modal LLM in open-source community 🎬🎙️ 🔥 Key Highlights: • Full-duplex Omni-modal Live Streaming: The model can see, listen, and speak simultaneously in a real-time conversation without mutual blocking • Proactive Interaction: Moving beyond reactive QA to performing proactive interaction, such as initiating reminders • Leading Performance: Scoring 77.6 on OpenCompass, it outperforms GPT-4o & Gemini 2.0 Pro in vision-language tasks with 9B params The best part? You can experience all above on your PC! #MiniCPM #OpenSource #MultimodalAI #LLM

OpenBMB

398,258 Aufrufe • vor 7 Monaten

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Charly Wargnier ♨️

566,171 Aufrufe • vor 8 Monaten

How far are we from "Her"? I couldn't help but wonder, given the rapid fire of GPT-Live and Thinking Machines. Turns out, full duplex speech-to-speech is not particularly new. Moshi has been around for more than 2 years. Many thanks to Neil Zeghidour for bringing so much nuance. Check out kyutai who are doing god's work and open-sourcing it (not sponsored!). 00:00 Intro 01:24 ChatGPT Standard & LLM-based cascades 04:08 ChatGPT Advanced & end-to-end speech models 05:35 ChatGPT Live & Her 07:38 Thinking Machines & multi-stream modeling 09:13 Moshi & full duplex assistants 13:24 Moshi’s inner monologue 15:38 Why aren’t full duplex models everywhere? 17:08 Tool calling in full duplex models 18:00 Moshi RAG 19:06 Is the future full duplex? 20:24 How far are we from Her?

How far are we from "Her"? I couldn't help but wonder, given the rapid fire of GPT-Live and Thinking Machines. Turns out, full duplex speech-to-speech is not particularly new. Moshi has been around for more than 2 years. Many thanks to Neil Zeghidour for bringing so much nuance. Check out kyutai who are doing god's work and open-sourcing it (not sponsored!). 00:00 Intro 01:24 ChatGPT Standard & LLM-based cascades 04:08 ChatGPT Advanced & end-to-end speech models 05:35 ChatGPT Live & Her 07:38 Thinking Machines & multi-stream modeling 09:13 Moshi & full duplex assistants 13:24 Moshi’s inner monologue 15:38 Why aren’t full duplex models everywhere? 17:08 Tool calling in full duplex models 18:00 Moshi RAG 19:06 Is the future full duplex? 20:24 How far are we from Her?

Julia Turc

73,293 Aufrufe • vor 2 Monaten

Breeze-TTS-2 is here. 🎙️ Real-time voice cloning, design, and direction in one model. 🤖 🏆 #1 open-weight TTS on Artificial Analysis, plus first place on voice design, instruction following, and latency benchmarks. 🎭 Clone a voice while preserving timbre, rhythm, emotion, and style, or design and direct new Chinese and English voices with natural-language prompts and inline vocal events. ⚡ Under 40 ms TTFA and 3.1× real-time streaming on the warmed-up fast path. Eager inference uses ~7.7 GiB, with a 12 GB GPU recommended. 📜 Code: Apache 2.0. Weights, derivatives, and self-hosted outputs: research and non-commercial use only.

Breeze-TTS-2 is here. 🎙️ Real-time voice cloning, design, and direction in one model. 🤖 🏆 #1 open-weight TTS on Artificial Analysis, plus first place on voice design, instruction following, and latency benchmarks. 🎭 Clone a voice while preserving timbre, rhythm, emotion, and style, or design and direct new Chinese and English voices with natural-language prompts and inline vocal events. ⚡ Under 40 ms TTFA and 3.1× real-time streaming on the warmed-up fast path. Eager inference uses ~7.7 GiB, with a 12 GB GPU recommended. 📜 Code: Apache 2.0. Weights, derivatives, and self-hosted outputs: research and non-commercial use only.

ModelScope

16,256 Aufrufe • vor 23 Tagen

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

kwindla

331,175 Aufrufe • vor 28 Tagen

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

OpenAI shipped a new speech-to-speech model today: gpt-realtime-2 This is the first speech-to-speech model good enough to use in my voice agents that do "real work." Or real play, for that matter. Here's gpt-realtime-2 as the brain of the ship AI in Gradient Bang. The voice-to-voice response and tool calling times here are unedited, so you can see exactly what the interaction with the model is like in an agent with a very complex system instruction and frequent tool calls. (I did clip out the subagent task execution segments, after gpt-realtime-2 starts a subagent via a tool call. Subagents in this config used gpt-5.2 "medium" effort.)

kwindla

55,222 Aufrufe • vor 4 Monaten

MiniCPM-o 4.5: Seeing, Listening, and Speaking — All at Once. 👁️👂🗣️ ✨Beyond traditional turn-taking, we’ve built a Native Full-Duplex engine that allows a 9B model to see, listen, and speak in one concurrent, non-blocking stream. Watch how it masters real-world complexity in real-time: 🔔 Proactive Auditory Interaction: Interrupts itself to alert you when it hears a "Ding!" while reading cards. 🎨 Temporal Flow Tracking: Follows your pen in real-time, narrating and "mind-reading" your drawing as you sketch. 🍎 Omni-Perception: Scans groceries & identifies prices on the fly. ✨Why it’s a category-leader: 📌Performance: Surpasses GPT-4o and Gemini 2.0 Pro on OpenCompass (Avg. 77.6). 📌Architecture: End-to-end fusion of SigLip2, Whisper, and CosyVoice2 on a Qwen3-8B base. 📌Efficiency: Full-duplex live streaming now runs locally on PCs via llama.cpp-omni. The era of "Wait-and-Response" AI is over. Proactive, real-time intelligence is now open-source. 🚀Experience it on Hugging Face: 🔗 #MiniCPM #Omnimodal #FullDuplex #EdgeAI #OpenSource #ComputerVision

MiniCPM-o 4.5: Seeing, Listening, and Speaking — All at Once. 👁️👂🗣️ ✨Beyond traditional turn-taking, we’ve built a Native Full-Duplex engine that allows a 9B model to see, listen, and speak in one concurrent, non-blocking stream. Watch how it masters real-world complexity in real-time: 🔔 Proactive Auditory Interaction: Interrupts itself to alert you when it hears a "Ding!" while reading cards. 🎨 Temporal Flow Tracking: Follows your pen in real-time, narrating and "mind-reading" your drawing as you sketch. 🍎 Omni-Perception: Scans groceries & identifies prices on the fly. ✨Why it’s a category-leader: 📌Performance: Surpasses GPT-4o and Gemini 2.0 Pro on OpenCompass (Avg. 77.6). 📌Architecture: End-to-end fusion of SigLip2, Whisper, and CosyVoice2 on a Qwen3-8B base. 📌Efficiency: Full-duplex live streaming now runs locally on PCs via llama.cpp-omni. The era of "Wait-and-Response" AI is over. Proactive, real-time intelligence is now open-source. 🚀Experience it on Hugging Face: 🔗 #MiniCPM #Omnimodal #FullDuplex #EdgeAI #OpenSource #ComputerVision

OpenBMB

115,537 Aufrufe • vor 7 Monaten

Today we’re releasing Chroma 1.0 → the world first open-source, end-to-end, real-time speech-to-speech model → with personalized voice cloning Trained by FlashLabs. Deployed on FlashAI👉 An open research-grade alternative to the OpenAI Realtime model. Voice Test dubbing Elon Musk and Lex Fridman: 🔥What’s real (evals and benchmarks attached): ⚡ 2× faster than real-time ) 📘 SGLang Cookbook: 📄 Paper + benchmarks: 🤗 Models: 💻 Inference code: 🔁 RT if you believe real-time AI should be open 💬 Reply if you’re building on Conversational Voice AI products

Today we’re releasing Chroma 1.0 → the world first open-source, end-to-end, real-time speech-to-speech model → with personalized voice cloning Trained by FlashLabs. Deployed on FlashAI👉 An open research-grade alternative to the OpenAI Realtime model. Voice Test dubbing Elon Musk and Lex Fridman: 🔥What’s real (evals and benchmarks attached): ⚡ 2× faster than real-time ) 📘 SGLang Cookbook: 📄 Paper + benchmarks: 🤗 Models: 💻 Inference code: 🔁 RT if you believe real-time AI should be open 💬 Reply if you’re building on Conversational Voice AI products

FlashLabs

1,249,149 Aufrufe • vor 8 Monaten

Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!

Voice agents still don’t understand who’s speaking to them. That’s a huge gap compared with humans, hidden by all the “phone-call” demos. But that changes today! NVIDIA is open-sourcing Nemotron 3 Diarization: a model that can reliably track speakers in live conversations, under a commercial-friendly license! In my tests, the quality is really good with one-second speech chunks. So we can use it for voice agents! I tested it with Reachy Mini and speech-to-speech running on a DGX Spark. It’s super fun to see the robot notice a new voice, ask for a name, and remember it. The model has day-zero integration with Transformers! Kudos to the NVIDIA team for shipping useful tools for the whole community!

Andi Marafioti

67,192 Aufrufe • vor 1 Tag

How did a tiny, scrappy team build one of the most powerful AI voice models? In a deep dive with Sesame CTO Ankit Kumar and a16z's Anjney Midha, we explore how Sesame is pushing the boundaries of AI conversation, why it open-sourced its speech generation model, and the power of small teams to outdo much larger AI labs on product focus. A part of their secret: a relentless focus on real-time, natural conversations over raw intelligence, and a deep commitment to voice, personality, and user experience. By opening up its speech generation model, Sesame is paving the way for even more breakthroughs in AI-native conversation 👇

How did a tiny, scrappy team build one of the most powerful AI voice models? In a deep dive with Sesame CTO Ankit Kumar and a16z's Anjney Midha, we explore how Sesame is pushing the boundaries of AI conversation, why it open-sourced its speech generation model, and the power of small teams to outdo much larger AI labs on product focus. A part of their secret: a relentless focus on real-time, natural conversations over raw intelligence, and a deep commitment to voice, personality, and user experience. By opening up its speech generation model, Sesame is paving the way for even more breakthroughs in AI-native conversation 👇

a16z

29,930 Aufrufe • vor 1 Jahr

What if your voice AI could interrupt you the moment it figured out your question - sometimes even before you finished asking it? Last week, I sat down with Neil, CEO of Gradium and co-founder of Kyutai , to talk about the future of speech-to-speech models and why he believes today's cascaded voice systems will soon look "archaic and brittle." Some highlights from our conversation: 🎯 How Kyutai built Moshi—a full duplex conversational AI with "negative latency"—in 6 months with just 4-6 people (while big tech teams had 10-20x the resources) 🧠 Why speech-to-speech models lose intelligence compared to their text counterparts (and what's being done about it) 📱 Pocket TTS: The first voice cloning model that runs on your phone's CPU—not GPU, CPU 🤖 Why robotics and spatial audio represent the next frontier (hint: current voice systems completely break in these environments) 👶 The efficiency gap: Babies learn to speak fluently from <5,000 hours of audio. Current models train on millions of hours. We're doing something wrong. My favorite vision from Neil? The first truly contrarian AI that interrupts you mid-sentence to tell you why you're wrong. Not just more natural conversation—but actually useful for testing ideas and playing devil's advocate. Full episode and detailed blog post linked in the comments 👇 What's your take - will speech-to-speech replace cascaded systems, or will modularity keep cascaded architectures dominant even as naturalness improves?

What if your voice AI could interrupt you the moment it figured out your question - sometimes even before you finished asking it? Last week, I sat down with Neil, CEO of Gradium and co-founder of Kyutai , to talk about the future of speech-to-speech models and why he believes today's cascaded voice systems will soon look "archaic and brittle." Some highlights from our conversation: 🎯 How Kyutai built Moshi—a full duplex conversational AI with "negative latency"—in 6 months with just 4-6 people (while big tech teams had 10-20x the resources) 🧠 Why speech-to-speech models lose intelligence compared to their text counterparts (and what's being done about it) 📱 Pocket TTS: The first voice cloning model that runs on your phone's CPU—not GPU, CPU 🤖 Why robotics and spatial audio represent the next frontier (hint: current voice systems completely break in these environments) 👶 The efficiency gap: Babies learn to speak fluently from <5,000 hours of audio. Current models train on millions of hours. We're doing something wrong. My favorite vision from Neil? The first truly contrarian AI that interrupts you mid-sentence to tell you why you're wrong. Not just more natural conversation—but actually useful for testing ideas and playing devil's advocate. Full episode and detailed blog post linked in the comments 👇 What's your take - will speech-to-speech replace cascaded systems, or will modularity keep cascaded architectures dominant even as naturalness improves?

Brooke Hopkins

13,000 Aufrufe • vor 7 Monaten

We’re excited to share that Nuance Labs has raised a $50M Series A led by Lightspeed, joined by Accel, South Park Commons, NVIDIA, and Define Ventures! Much of human communication happens beyond words. Before we can speak, we learn to read faces, return smiles, follow gestures, and sense when someone is listening. That visual and emotional vocabulary stays with us for life. At Nuance Labs, we’re building a foundation model that understands and responds to the full spectrum of human emotion. A single full-duplex audio-visual model that sees, listens, and responds in real time, with the natural give-and-take of a conversation with a friend or colleague. We believe the best interface with a machine is the one we’ve practiced since birth. Join us!

We’re excited to share that Nuance Labs has raised a $50M Series A led by Lightspeed, joined by Accel, South Park Commons, NVIDIA, and Define Ventures! Much of human communication happens beyond words. Before we can speak, we learn to read faces, return smiles, follow gestures, and sense when someone is listening. That visual and emotional vocabulary stays with us for life. At Nuance Labs, we’re building a foundation model that understands and responds to the full spectrum of human emotion. A single full-duplex audio-visual model that sees, listens, and responds in real time, with the natural give-and-take of a conversation with a friend or colleague. We believe the best interface with a machine is the one we’ve practiced since birth. Join us!

Fangchang Ma

2,002,817 Aufrufe • vor 10 Tagen

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

Lior Alexander

61,122 Aufrufe • vor 8 Monaten

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

atomic.chat

113,185 Aufrufe • vor 1 Monat

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

kwindla

40,593 Aufrufe • vor 4 Monaten

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Base Labs

236,375 Aufrufe • vor 20 Tagen

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

OpenBMB

13,996 Aufrufe • vor 8 Monaten

QVAC SDK 0.13.0 is live, and this version brings a lot of exciting updates! Local AI now plugs into your coding agent, ships as a desktop app in one command, and runs even more models. Highlights: NEW INTEGRATIONS - OpenCode and coding agents: the new @qvac/ai-sdk-provider makes QVAC a local provider. Less setup, same-model requests queue cleanly, and managed mode starts and supervises qvac serve for you. - Broader OpenAI-compatible API, validated across supported flows so covered capabilities stay consistent and testable. - Turn your QVAC project into a real desktop app for Mac, Windows, or Linux with a single command. The new Electron plugin handles the packaging and keeps the app small by including only what it needs. NEW MODELS - New pi0.5 model support - run a vision-language "robot brain" on a single ordinary graphics card, at full accuracy. - Image-to-video, fully local, via the Wan2.1 model in the Diffusion addon. - New BCI add-on: brain-computer interface transcription, fully local. Decode recorded neural signals into text on-device via the Whisper.cpp-based BCI model. IMPROVEMENTS - Whisper GPU transcription on Android, auto-picking the best backend (OpenCL on Adreno 700+, Vulkan elsewhere), unified on one ggml engine. - Parakeet steadier on mobile, with real end-of-utterance detection for streaming. - Supertonic TTS now runs full GPU across Metal, Vulkan, and OpenCL, with native streaming.

QVAC SDK 0.13.0 is live, and this version brings a lot of exciting updates! Local AI now plugs into your coding agent, ships as a desktop app in one command, and runs even more models. Highlights: NEW INTEGRATIONS - OpenCode and coding agents: the new @qvac/ai-sdk-provider makes QVAC a local provider. Less setup, same-model requests queue cleanly, and managed mode starts and supervises qvac serve for you. - Broader OpenAI-compatible API, validated across supported flows so covered capabilities stay consistent and testable. - Turn your QVAC project into a real desktop app for Mac, Windows, or Linux with a single command. The new Electron plugin handles the packaging and keeps the app small by including only what it needs. NEW MODELS - New pi0.5 model support - run a vision-language "robot brain" on a single ordinary graphics card, at full accuracy. - Image-to-video, fully local, via the Wan2.1 model in the Diffusion addon. - New BCI add-on: brain-computer interface transcription, fully local. Decode recorded neural signals into text on-device via the Whisper.cpp-based BCI model. IMPROVEMENTS - Whisper GPU transcription on Android, auto-picking the best backend (OpenCL on Adreno 700+, Vulkan elsewhere), unified on one ggml engine. - Parakeet steadier on mobile, with real end-of-utterance detection for streaming. - Supertonic TTS now runs full GPU across Metal, Vulkan, and OpenCL, with native streaming.

QVAC

20,932,195 Aufrufe • vor 3 Monaten