Every millisecond counts for voice agents. Introducing Gemma 4 31B on LiveKit Inference! Optimized for real-time voice agents: - 354ms time to first audio - 192ms time to first token - Beats GPT-4.1 in agentic tool use (31B achieves 76.9% on tau2bench) Fast, capable, and efficient!

Did he just said gpt-4.1?

Learn more and test the integration right here:

354 milliseconds and it still replies faster than my group chat

my college wifi has higher latency than that 354ms, i'm officially moving into your data center.

I swear the Gemma team has its shit more together than any google team deploying gemini products.

This seems to be a real use case battle, flying a little bit under the radar. While everyone is looking at general models, like Fable this, Codex that, voice agents play in a world of their own. Curious to see who gets the biggest market slice here.

There was a GPT 4.1? How long ago was that?

gpt what

It would be cool to test Gemma again, maybe you could bring back unlimited TPM in the API? Because the model is currently dead.

In the architecture of conversation, every millisecond is a bridge toward true connection. By narrowing the gap between thought and response, these optimizations do more than just improve speed—they cultivate a more fluid, human-centric experience. It is a remarkable step toward making technology feel less like a tool and more like a natural extension of our daily interactions. 🎙️✨

Super model for reachy mini

i just use Gemma-4-12b with direct audio and toolcall. leapfrogging traditional STT->LLM->TTS pipeline.

It's useful for developers..

Voice model development matters more than people give it credit for. Think about every call center on earth right now. "You are 18th in the queue" is one of the most universally hated experiences in customer service, and it's completely solvable at this point. ElevenLabs already proved the use case works, real-time voice agents in major languages, built on genuinely human-sounding voices. I know firsthand that a well-built voice agent delivers a good, actually good, customer experience, not the robotic phone-tree nightmare people expect. This is exactly the kind of problem that needs solving, and a fast, capable model like this is another piece of the puzzle. 354ms to first audio is the number that actually matters here, that's the difference between something feeling responsive and something feeling broken.

OK to be clear, the audio and the text are coming from the same model with shared context? And is audio input part of the context, and if so, is it transcribed first, or does the model capture extra 'nuance' from the audio input? Or is the audio coming from a separate tool?

But have you seen K3 !?

Wow you guys beat GPT-4.1!!! WOWWW!!!

Gemma 4.5 please

354ms to first audio is actually insane for a voice agent real time AI convos about to feel genuinely human

Livekit has inference as a service? Lol. On it

354ms to first audio is wild. is that median or p95, and does it include endpointing?

The future of voice AI won't be won by the smartest model—it'll be won by the model that feels the most natural.

192ms to first token matters more than the benchmark slide for voice agents. Latency is the UX.

Every millisecond counts for voice agents. Every missed update counts for Discord communities. Sage Bot keeps activity, announcements, and engagement moving in real time.🫡

354ms for a voice agent is wild ngl but does it still go 'uh... one moment' mid sentence 💀

Can it beat Kimi 3? This how open source model should be

What is the required hardware to run it with local inference at these speeds?

¿Maneja mejor el uso de tools con agentes tipo pi u opencode?

Beating GPT 4.1 on tool use at that size is genuinely impressive.

Impressive latency numbers. Curious how this compares to a distilled Whisper + small LLM running on-device for latency-sensitive voice apps. The cloud vs edge tradeoff for voice agents is still wide open.

Sub 200ms time to first token is a big deal for voice agents

354ms is wild but the real test is when grandma asks it something mid-sentence and it doesnt lose the thread

192ms first token on 31B is wild for voice agents. Real-time latency was the bottleneck — this changes production deployments

CodeSTACKr 😎