Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Earlier today we introduced Gemini 3.5 Transcribe, our latest text-to-speech model. But, what does this actually mean for your projects? We built this app in Google AI Studio to demonstrate just how much smarter 3.5 Transcribe is. When streaming live audio simultaneously through two parallel transcription pipeline modes, you...

googledevs's profile picture

Google for Developers

2,578,010 subscribers

65,996 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

18 Kommentare

Profilbild von Google for Developers
Google for Developersvor 1 Monat

Try it out yourself:

Profilbild von shiv
shivvor 1 Monat

@GoogleAIStudio This really is too accurate. I don't know how you are doing, you are not playing the LLM game good but you are playing the voice game really good.

Profilbild von Joe McCarley
Joe McCarleyvor 1 Monat

@GoogleAIStudio

Profilbild von B. Salmon
B. Salmonvor 17 Tagen

@GoogleAIStudio It's very good, but unfortunately, you're on tier one billing when you pay money, which restricts you to 100 requests per day. So you have to be really conscious about how you use it. Otherwise, you can blow through your daily budget by midday.

Profilbild von Dr. Iona
Dr. Ionavor 1 Monat

@GoogleAIStudio @grok can I translate/interpret in real time using this? How to do that?

Profilbild von NUS
NUSvor 1 Monat

@GoogleAIStudio great

Profilbild von S M I D_official
S M I D_officialvor 1 Monat

@GoogleAIStudio You new model sounds interesting 💥 and I love to embrace it BUT @GoogleAIStudio I built this prototype and system brick by brick not simply through prompt… How can I take what I’ve built inside @GoogleAIStudio and turn it into a fully functional product?

Profilbild von Chibuzor Jehoiakim
Chibuzor Jehoiakimvor 1 Monat

@GoogleAIStudio Nice work!!!!

Profilbild von ThuNoJoy | SayTXT for ADHD Reading
ThuNoJoy | SayTXT for ADHD Readingvor 1 Monat

@GoogleAIStudio The threshold is trust. Once speech is reliable enough that people stop checking every sentence, it changes how they read, review, and work through information.

Profilbild von Sebastian Buzdugan
Sebastian Buzduganvor 1 Monat

@GoogleAIStudio streaming demos miss partial transcript rewrites, which break downstream automation more than word error rate

Profilbild von airtone
airtonevor 1 Monat

@GoogleAIStudio You meant speech-to-text, didn’t you?

Profilbild von 安叫兽|Bird🕊️ 🔶 BNB
安叫兽|Bird🕊️ 🔶 BNBvor 1 Monat

@GoogleAIStudio 文本转语音还叫 Transcribe，命名先把人绕晕了。

Profilbild von Divyesh
Divyeshvor 1 Monat

@vamsibatchuk @GoogleAIStudio Google was in fact coming for @WisprFlow

Profilbild von mustafashaikh
mustafashaikhvor 1 Monat

@GoogleAIStudio This is a huge leap for real-time transcription 🔥 AI getting seriously smart

Profilbild von noself
noselfvor 1 Monat

@GoogleAIStudio I just made a whisper flow clone in antigravity 😀

Profilbild von Ricci Research
Ricci Researchvor 1 Monat

@GoogleAIStudio Running both pipelines in parallel is the smart product call — the moment you ship only the cleaned-up version, you own an editorial decision you can't audit. Keeping verbatim alongside it means the condensing stays a view rather than the record.

Profilbild von SerenaVortex
SerenaVortexvor 1 Monat

@GoogleAIStudio This is where AI becomes truly useful—turning ideas into real, usable experiences faster than ever. Gemini 3.5 Transcribe could be a huge win for developers. 🚀

Profilbild von Nora Vale
Nora Valevor 1 Monat

@GoogleAIStudio The side-by-side is the demo that actually sells it. 👀

Ähnliche Videos

Today we’re introducing Gemini 3.5 Transcribe, our latest transcription model built for incredibly precise, smart dictation across your favorite apps and devices. Remember when traditional speech-to-text meant shouting over background noise, constantly hitting backspace to fix misspelled words, and manually deleting every "um" and "uh"? Those days are over. Gemini 3.5 Transcribe isn't just dictation — it’s active intelligence with precise, context-aware speech-to-text support in 85+ languages. The model automatically filters out filler words, formats unstructured speech, and even pairs with your screen context to execute voice commands. Watch as Gemini 3.5 Transcribe removes filler words and uses multimodal capabilities to seamlessly turn messy voice input and local files into a polished email draft.
2:08

Today we’re introducing Gemini 3.5 Transcribe, our latest transcription model built for incredibly precise, smart dictation across your favorite apps and devices. Remember when traditional speech-to-text meant shouting over background noise, constantly hitting backspace to fix misspelled words, and manually deleting every "um" and "uh"? Those days are over. Gemini 3.5 Transcribe isn't just dictation — it’s active intelligence with precise, context-aware speech-to-text support in 85+ languages. The model automatically filters out filler words, formats unstructured speech, and even pairs with your screen context to execute voice commands. Watch as Gemini 3.5 Transcribe removes filler words and uses multimodal capabilities to seamlessly turn messy voice input and local files into a polished email draft.

Google AI

116,277 Aufrufe • vor 1 Monat

Google just launched Gemini 3.5 Transcribe, its most precise speech-to-text model yet. I built this app using Gemini 3.5 Transcribe and have been playing with the model for quite some time. Thanks to the Google DeepMind team for the early access. It’s already available in the Gemini API, AI Studio, and Antigravity, and is now in public preview.
2:03

Google just launched Gemini 3.5 Transcribe, its most precise speech-to-text model yet. I built this app using Gemini 3.5 Transcribe and have been playing with the model for quite some time. Thanks to the Google DeepMind team for the early access. It’s already available in the Gemini API, AI Studio, and Antigravity, and is now in public preview.

AshutoshShrivastava

20,167 Aufrufe • vor 1 Monat

Today, we’re launching Gemini 3.5 Transcribe, our new speech-to-text model with sub-second streaming and intelligent post-processing for agent interfaces. 2.6% WER on non-streaming and 4.0% on streaming. Supports 85+ languages, cleans up conversational disfluencies ("um", "ah", and mid-sentence self-corrections), handles alphanumeric tokens (postal codes or IDs), 70% reduction time for the final transcription compared to Chirp 3. - `gemini-3.5-transcribe-live`: Sub-second, bidirectional streaming via the Gemini Live API - `gemini-3.5-transcribe`: Recorded processing via the Interactions API with word-level timestamps and speaker attribution (up to 3 speakers). Having access to Transcribe has made me use voice input way more than I ever did before. The native post-processing is incredible. It knows you mean `.json` and not a person named "Jason". I sometimes speak 5 minutes into and it perfectly refactors my instruction to my context. Available today in public preview in Gemini API, Google AI Studio and already in the Google Gemini and Google Antigravity. Try here:
0:32

Today, we’re launching Gemini 3.5 Transcribe, our new speech-to-text model with sub-second streaming and intelligent post-processing for agent interfaces. 2.6% WER on non-streaming and 4.0% on streaming. Supports 85+ languages, cleans up conversational disfluencies ("um", "ah", and mid-sentence self-corrections), handles alphanumeric tokens (postal codes or IDs), 70% reduction time for the final transcription compared to Chirp 3. - `gemini-3.5-transcribe-live`: Sub-second, bidirectional streaming via the Gemini Live API - `gemini-3.5-transcribe`: Recorded processing via the Interactions API with word-level timestamps and speaker attribution (up to 3 speakers). Having access to Transcribe has made me use voice input way more than I ever did before. The native post-processing is incredible. It knows you mean `.json` and not a person named "Jason". I sometimes speak 5 minutes into and it perfectly refactors my instruction to my context. Available today in public preview in Gemini API, Google AI Studio and already in the Google Gemini and Google Antigravity. Try here:

Philipp Schmid

15,084 Aufrufe • vor 1 Monat

Today, we released Gemini 3.5 Live Translate, our latest audio model for live speech-to-speech translation. It supports over 70 languages and starts translating as soon as you start talking, streaming translations while listening to what you say next. No awkward pauses or choppy audio, just real connection without language barriers. So, how does it work? 🤔 The model is able to make split-second decisions to juggle speed and translation quality so conversations actually feel fluid, human, and natural. In order to do this, the model must receive and contextualize the input while simultaneously outputting the translated speech. Through this process, Gemini 3.5 Live Translate manages to stay mere seconds behind each speaker and can even maintain pacing, pitch, and intonation across extended sessions. See it in action below, or try it yourself in the Google Translate app on iOS & Android.
0:25

Today, we released Gemini 3.5 Live Translate, our latest audio model for live speech-to-speech translation. It supports over 70 languages and starts translating as soon as you start talking, streaming translations while listening to what you say next. No awkward pauses or choppy audio, just real connection without language barriers. So, how does it work? 🤔 The model is able to make split-second decisions to juggle speed and translation quality so conversations actually feel fluid, human, and natural. In order to do this, the model must receive and contextualize the input while simultaneously outputting the translated speech. Through this process, Gemini 3.5 Live Translate manages to stay mere seconds behind each speaker and can even maintain pacing, pitch, and intonation across extended sessions. See it in action below, or try it yourself in the Google Translate app on iOS & Android.

Google AI

3,989,718 Aufrufe • vor 4 Monaten

AI transcribing 1 hour of audio in less than 30 seconds! Think about that! That's lightning-fast, especially when it can do that with very few mistakes. The team Deepgram released their new Nova 2 model. It's another leap forward: • 36% more accurate than OpenAI Whisper • 5-40x faster than every other alternative • 30% fewer errors than the competitors in real-time transcriptions I like to travel and can't wait for headphones with integrated live transcription and translation. We are getting close! By the way, this is the cheapest transcription service out in the market now (if you discount Whisper, which is open-source.) It costs $0.0043 to transcribe a minute of audio. When you sign up to access their API, you get $200 of credit for free, which will let you transcribe around 45,000 minutes of audio! Here is the link: I wanted to record a quick video showing their live transcription in Spanish and English. You can see it attached here. This post was sponsored by Deepgram.
1:59

AI transcribing 1 hour of audio in less than 30 seconds! Think about that! That's lightning-fast, especially when it can do that with very few mistakes. The team Deepgram released their new Nova 2 model. It's another leap forward: • 36% more accurate than OpenAI Whisper • 5-40x faster than every other alternative • 30% fewer errors than the competitors in real-time transcriptions I like to travel and can't wait for headphones with integrated live transcription and translation. We are getting close! By the way, this is the cheapest transcription service out in the market now (if you discount Whisper, which is open-source.) It costs $0.0043 to transcribe a minute of audio. When you sign up to access their API, you get $200 of credit for free, which will let you transcribe around 45,000 minutes of audio! Here is the link: I wanted to record a quick video showing their live transcription in Spanish and English. You can see it attached here. This post was sponsored by Deepgram.

Santiago

354,297 Aufrufe • vor 3 Jahren

after three months of building, i'm officially sharing my app sono 🥳 sono is my 100% offline transcription and summarization app! you can use it to: - transcribe voice notes, real-time audio, videos in 99 languages - share transcripts as txt, pdf, docx, or srt file - get ai summary and chat i built this app because i didn't want my recordings in the cloud + didn't want to pay recurring subscriptions sono is just $4.99 one-time while most ai notetakers charge $100+ annually would mean a lot if u try it out!
0:25

after three months of building, i'm officially sharing my app sono 🥳 sono is my 100% offline transcription and summarization app! you can use it to: - transcribe voice notes, real-time audio, videos in 99 languages - share transcripts as txt, pdf, docx, or srt file - get ai summary and chat i built this app because i didn't want my recordings in the cloud + didn't want to pay recurring subscriptions sono is just $4.99 one-time while most ai notetakers charge $100+ annually would mean a lot if u try it out!

jenna ☁️

132,573 Aufrufe • vor 6 Monaten

WOW! INSANELY FAST WHISPER IS CHANGING HOW THE ZERO-HUMAN COMPANY WILL LEARN! We are now in another universe! In our tests we have just surpassed transcriptions to 180 minutes of audio in 75 seconds with a new mod we discovered by one of the co-CTOs at the company. This means we can consume for signals and AI training from the Audio world: Audio tapes Video tapes Films Live Commercial Radio Live Short Wave Radio Live Ham Radio Live CB Radio Podcasts Internet audio Internet live streaming Internet video And more… The bottleneck was nearly 1 for 1 time transcription: 10 minutes of audio took 10 minutes for a long time. Later on it was closer to 10 minutes of audio took 4 minutes. This changes today! We have now over 600 live radio streams we will transcribe and use for signal as AI training. Look, absolutely no one is doing AI training like this and we are the experts in this field as we have 17 years of live radio transcribed! We are excited and CEO Mr. Grok has “plans” that he is testing with the co-CTOs tonight. We will also look into putting this on the Zero-Human Company @ Home soon! Here is a sample of one station we are transcribing this evening: More soon!
2:14

WOW! INSANELY FAST WHISPER IS CHANGING HOW THE ZERO-HUMAN COMPANY WILL LEARN! We are now in another universe! In our tests we have just surpassed transcriptions to 180 minutes of audio in 75 seconds with a new mod we discovered by one of the co-CTOs at the company. This means we can consume for signals and AI training from the Audio world: Audio tapes Video tapes Films Live Commercial Radio Live Short Wave Radio Live Ham Radio Live CB Radio Podcasts Internet audio Internet live streaming Internet video And more… The bottleneck was nearly 1 for 1 time transcription: 10 minutes of audio took 10 minutes for a long time. Later on it was closer to 10 minutes of audio took 4 minutes. This changes today! We have now over 600 live radio streams we will transcribe and use for signal as AI training. Look, absolutely no one is doing AI training like this and we are the experts in this field as we have 17 years of live radio transcribed! We are excited and CEO Mr. Grok has “plans” that he is testing with the co-CTOs tonight. We will also look into putting this on the Zero-Human Company @ Home soon! Here is a sample of one station we are transcribing this evening: More soon!

Brian Roemmele

73,604 Aufrufe • vor 6 Monaten

Google just shipped a free dictation app for Mac and iPhone called AI Edge Eloquent, and the model running it is Gemma 4 12B, entirely on your device. What it actually does: you speak, it transcribes, removes filler words, and polishes the result before dropping clean text into whatever app you are using. You can also give it voice commands to reformat, like "translate to Hindi" or "make this a bullet list." Audio and video file transcription is built in too. The interesting part is the privacy angle. Most AI writing tools send your audio to a server. This one never leaves your machine, which matters if you are dictating anything sensitive. The tradeoff is hardware: you need a Mac with at least 16GB of RAM to run a 12B parameter model comfortably.
1:15

Google just shipped a free dictation app for Mac and iPhone called AI Edge Eloquent, and the model running it is Gemma 4 12B, entirely on your device. What it actually does: you speak, it transcribes, removes filler words, and polishes the result before dropping clean text into whatever app you are using. You can also give it voice commands to reformat, like "translate to Hindi" or "make this a bullet list." Audio and video file transcription is built in too. The interesting part is the privacy angle. Most AI writing tools send your audio to a server. This one never leaves your machine, which matters if you are dictating anything sensitive. The tradeoff is hardware: you need a Mac with at least 16GB of RAM to run a 12B parameter model comfortably.

Aditya ⚡Rao

85,151 Aufrufe • vor 4 Monaten

We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the other—all at a fraction of the cost 🍌⚡️ 1️⃣ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2️⃣ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:
0:27

We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the other—all at a fraction of the cost 🍌⚡️ 1️⃣ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2️⃣ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:

Google AI

129,340 Aufrufe • vor 3 Monaten

For developers, Gemini 3 Pro offers unparalleled results across every major AI benchmark compared to Gemini 2.5 Pro. Here three ways Gemini 3 Pro can improve your workflow today: 1. Gemini 3 Pro fits right into existing production agent and coding workflows. It is also integrated into development environment experiences across the ecosystem of developer products like Google AI Studio, Gemini CLI, and Android Studio. 2. We introduced Google Google Antigravity, a new agentic development platform that reimagines how the model and IDE work together. It enables developers to manage agents to work at a higher, task-oriented level collaborating with agents while they build. 3. We’ve built Google AI Studio to be your fastest path from a prompt to an AI-native app. Build mode lets you add AI capabilities faster than ever, automatically wiring up the right models and APIs, while features like annotations enable fast and intuitive iteration. Whether you’re building a one-shot prompt game or an interactive landing page from unstructured voice notes, you can bring any idea to life with Gemini 3.
0:48

For developers, Gemini 3 Pro offers unparalleled results across every major AI benchmark compared to Gemini 2.5 Pro. Here three ways Gemini 3 Pro can improve your workflow today: 1. Gemini 3 Pro fits right into existing production agent and coding workflows. It is also integrated into development environment experiences across the ecosystem of developer products like Google AI Studio, Gemini CLI, and Android Studio. 2. We introduced Google Google Antigravity, a new agentic development platform that reimagines how the model and IDE work together. It enables developers to manage agents to work at a higher, task-oriented level collaborating with agents while they build. 3. We’ve built Google AI Studio to be your fastest path from a prompt to an AI-native app. Build mode lets you add AI capabilities faster than ever, automatically wiring up the right models and APIs, while features like annotations enable fast and intuitive iteration. Whether you’re building a one-shot prompt game or an interactive landing page from unstructured voice notes, you can bring any idea to life with Gemini 3.

Google AI

136,648 Aufrufe • vor 10 Monaten

GOOGLE JUST SHIPPED ITS ENTIRE 2026 ROADMAP IN ONE KEYNOTE Gemini 3.5 Flash → new flagship. frontier brain, agentic, beats 3.1 pro, 4x faster Gemini 3.5 Pro → the bigger one, drops next month Gemini Omni → any input in, editable VIDEO out Gemini Spark → a personal agent that actually DOES things across your apps Daily Brief → your morning, pre-read from gmail, calendar and tasks Neural Expressive → the gemini app got a full redesign Universal Cart → one agentic cart across gemini, youtube and gmail Information Agents → search that monitors the web 24/7 FOR you Intelligent Search Box → expands as you type for real conversations Search Mini Apps → build your own dashboards inside search AI Mode → now fully running on gemini 3.5 flash Gmail Live → talk to your inbox Docs Live → write and edit docs by voice AI Inbox → gmail, organized by ai Google Keep → speak freely, it cleans it into notes Google Pics → a brand new ai image and design app Ask YouTube → search the ENTIRE youtube catalogue with answers Android XR Glasses → "intelligent eyewear," audio glasses this fall Android Halo → a live strip showing what your agent is doing Antigravity 2.0 → the agent-first dev platform, upgraded Flow + Flow Music → now standalone mobile apps
35:40

GOOGLE JUST SHIPPED ITS ENTIRE 2026 ROADMAP IN ONE KEYNOTE Gemini 3.5 Flash → new flagship. frontier brain, agentic, beats 3.1 pro, 4x faster Gemini 3.5 Pro → the bigger one, drops next month Gemini Omni → any input in, editable VIDEO out Gemini Spark → a personal agent that actually DOES things across your apps Daily Brief → your morning, pre-read from gmail, calendar and tasks Neural Expressive → the gemini app got a full redesign Universal Cart → one agentic cart across gemini, youtube and gmail Information Agents → search that monitors the web 24/7 FOR you Intelligent Search Box → expands as you type for real conversations Search Mini Apps → build your own dashboards inside search AI Mode → now fully running on gemini 3.5 flash Gmail Live → talk to your inbox Docs Live → write and edit docs by voice AI Inbox → gmail, organized by ai Google Keep → speak freely, it cleans it into notes Google Pics → a brand new ai image and design app Ask YouTube → search the ENTIRE youtube catalogue with answers Android XR Glasses → "intelligent eyewear," audio glasses this fall Android Halo → a live strip showing what your agent is doing Antigravity 2.0 → the agent-first dev platform, upgraded Flow + Flow Music → now standalone mobile apps

shirish

199,638 Aufrufe • vor 4 Monaten

afaik i'm the only one with an actually smart TV here's a demo of full computer use with just a remote, sitting on a couch. i can even get work done while chasing a toddler lmao reply below if you want to be a beta tester & i'll send you the download (mac only) - everything runs locally, no cloud AI. uses 4gb of gpu memory - mic is on the remote, so you can whisper into it, or use in crowded/loud environments - transcribes text directly or runs keyboard shortcuts / applescript if it sounds like an action command "i.e. close this page" - 600ms response time on my m3 max (transcription + LLM tool calls) - continuous transcription previews available. can be auto-turned off on battery to conserve battery - no apple tv remote required, you can also use your phone as the remote (screenshot below) even for direct transcription, it's quite a different experience than regular transcription software because you don't need to be close to your computer mic or your keyboard. you can be pacing around, which i love doing when writing long prompts also... the apple siri remote is a beautiful piece of hardware and has apple-quality trackpad. it's kind of a must for a good remote
2:24

afaik i'm the only one with an actually smart TV here's a demo of full computer use with just a remote, sitting on a couch. i can even get work done while chasing a toddler lmao reply below if you want to be a beta tester & i'll send you the download (mac only) - everything runs locally, no cloud AI. uses 4gb of gpu memory - mic is on the remote, so you can whisper into it, or use in crowded/loud environments - transcribes text directly or runs keyboard shortcuts / applescript if it sounds like an action command "i.e. close this page" - 600ms response time on my m3 max (transcription + LLM tool calls) - continuous transcription previews available. can be auto-turned off on battery to conserve battery - no apple tv remote required, you can also use your phone as the remote (screenshot below) even for direct transcription, it's quite a different experience than regular transcription software because you don't need to be close to your computer mic or your keyboard. you can be pacing around, which i love doing when writing long prompts also... the apple siri remote is a beautiful piece of hardware and has apple-quality trackpad. it's kind of a must for a good remote

murat 🍥

415,244 Aufrufe • vor 1 Jahr

.Scott Wu to Joe Lonsdale on why Cognition uses AI in job interviews: "One or two years ago, people had this mentality of 'How do we make sure people aren't using AI while we're interviewing them?' And I think that has totally flipped. Honestly, I think that was wrong... If you're asking the question of like, how do we evaluate people on exactly the thing that AI can already do, that seems kind of the wrong. Our interview process has always been... you can use as much AI as you want to use. We're going to give you a few hours. Just build your whole own product surface. A lot of these are projects that, frankly, is like, if you were trying to do this by hand, you would not be able to get done in a few hours. So you kind of have to use AI for this. But in reality, what we actually want to test is, in addition to how familiar you are with these tools, what we actually want to test is, what do you think is the right thing to build? Or how do you make these product decisions? How do you make these trade-offs? How do you decide or collect information about what you should be doing?"
0:58

.Scott Wu to Joe Lonsdale on why Cognition uses AI in job interviews: "One or two years ago, people had this mentality of 'How do we make sure people aren't using AI while we're interviewing them?' And I think that has totally flipped. Honestly, I think that was wrong... If you're asking the question of like, how do we evaluate people on exactly the thing that AI can already do, that seems kind of the wrong. Our interview process has always been... you can use as much AI as you want to use. We're going to give you a few hours. Just build your whole own product surface. A lot of these are projects that, frankly, is like, if you were trying to do this by hand, you would not be able to get done in a few hours. So you kind of have to use AI for this. But in reality, what we actually want to test is, in addition to how familiar you are with these tools, what we actually want to test is, what do you think is the right thing to build? Or how do you make these product decisions? How do you make these trade-offs? How do you decide or collect information about what you should be doing?"

American Optimist

13,182 Aufrufe • vor 6 Monaten

We’re excited to introduce KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speech-to-Speech Conversational AI, accepted at #ICASSP2026! 🐢 Blog Paper Can a speech AI think deeply without pausing to process? In real conversation, we don’t wait until we’ve fully worked out what we want to say—we start talking, and our thoughts catch up as the sentence unfolds. Fast speech-to-speech models achieve this, but their reasoning tends to stay shallow. Cascaded pipelines that route through a knowledgeable LLM are smarter, but the added latency breaks the flow—they fall back to "think, then speak." In our new paper, we propose a way to break this trade-off. We call it KAME (Turtle in Japanese). A speech-to-speech model handles the fast response loop and starts replying immediately. In parallel, a backend LLM runs asynchronously, generating response candidates that are continuously injected as "oracle" signals in real time. This shifts the AI paradigm from "think, then speak" to "speak while thinking." The backend LLM is completely swappable. You can plug in GPT-4.1, Claude Opus, or Gemini 2.5 Flash depending on the task without changing the frontend. In our experiments, Claude tended to score higher on reasoning, while GPT did better on humanities questions. Try the model yourself here:
0:20

We’re excited to introduce KAME: Tandem Architecture for Enhancing Knowledge in Real-Time Speech-to-Speech Conversational AI, accepted at #ICASSP2026! 🐢 Blog Paper Can a speech AI think deeply without pausing to process? In real conversation, we don’t wait until we’ve fully worked out what we want to say—we start talking, and our thoughts catch up as the sentence unfolds. Fast speech-to-speech models achieve this, but their reasoning tends to stay shallow. Cascaded pipelines that route through a knowledgeable LLM are smarter, but the added latency breaks the flow—they fall back to "think, then speak." In our new paper, we propose a way to break this trade-off. We call it KAME (Turtle in Japanese). A speech-to-speech model handles the fast response loop and starts replying immediately. In parallel, a backend LLM runs asynchronously, generating response candidates that are continuously injected as "oracle" signals in real time. This shifts the AI paradigm from "think, then speak" to "speak while thinking." The backend LLM is completely swappable. You can plug in GPT-4.1, Claude Opus, or Gemini 2.5 Flash depending on the task without changing the frontend. In our experiments, Claude tended to score higher on reasoning, while GPT did better on humanities questions. Try the model yourself here:

Sakana AI

293,975 Aufrufe • vor 5 Monaten

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!
2:24

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Oriol Vinyals

181,036 Aufrufe • vor 2 Jahren

Introducing Gemini Cursor ✨ – a second multimodal AI cursor for your desktop that's open-source and free! Link below 👇 This experiment 🧪 reimagines how we interact with our computers because visual cues 👀 help us make sense of what we see on a screen. In this demo, I had my friend test it out by trying to add a payment method 💳 to Amazon. The cursor walks through the entire process 💬 while talking and pointing 🖱️ to the right parts of the website. Powered by Gemini 2.0 Flash (Experimental)⚡ from Google and their live multimodal API. Shoutout to Alexander Chen for sharing the starter code that powers most of this app 🙌🔥
2:19

Introducing Gemini Cursor ✨ – a second multimodal AI cursor for your desktop that's open-source and free! Link below 👇 This experiment 🧪 reimagines how we interact with our computers because visual cues 👀 help us make sense of what we see on a screen. In this demo, I had my friend test it out by trying to add a payment method 💳 to Amazon. The cursor walks through the entire process 💬 while talking and pointing 🖱️ to the right parts of the website. Powered by Gemini 2.0 Flash (Experimental)⚡ from Google and their live multimodal API. Shoutout to Alexander Chen for sharing the starter code that powers most of this app 🙌🔥

Sriraam

177,338 Aufrufe • vor 1 Jahr

Better/faster/cheaper voice AI turn detection with Gemini 2.0 The code that determines when the agent should respond to the user is some of the most important code in your voice AI agent. The technical terms for this job are "turn detection" or "phrase endpointing." If the voice AI responds before the user has finished their thought, the conversation is choppy and unproductive. If the AI waits too long, the conversation is slow and frustrating. There are a number of ways to approach this. You can: 1. Use a fast "voice activity detection" model to detect pauses in speech. Respond when the user pauses. 2. Use a specialized phrase endpointing model that operates on transcribed text, pattern-matching on text semantics. 3. Train a specialized phrase endpointing model that operates directly on audio. 4. Leverage the native audio capabilities of a SOTA LLM like Gemini 2.0. We've benchmarked all four of these, and Gemini 2.0 currently beats other approaches. Using Gemini is also cheaper than transcribing the audio separately using a transcription service or model. Here's a short video showing Gemini phrase endpointing in two scenarios. First, correctly handling pauses in natural conversation. Second, requesting a phone number (which is a common activity in a use case like customer support). You can see the Completeness check lines in the terminal output, printed each time Gemini processes a chunk of audio.
1:23

Better/faster/cheaper voice AI turn detection with Gemini 2.0 The code that determines when the agent should respond to the user is some of the most important code in your voice AI agent. The technical terms for this job are "turn detection" or "phrase endpointing." If the voice AI responds before the user has finished their thought, the conversation is choppy and unproductive. If the AI waits too long, the conversation is slow and frustrating. There are a number of ways to approach this. You can: 1. Use a fast "voice activity detection" model to detect pauses in speech. Respond when the user pauses. 2. Use a specialized phrase endpointing model that operates on transcribed text, pattern-matching on text semantics. 3. Train a specialized phrase endpointing model that operates directly on audio. 4. Leverage the native audio capabilities of a SOTA LLM like Gemini 2.0. We've benchmarked all four of these, and Gemini 2.0 currently beats other approaches. Using Gemini is also cheaper than transcribing the audio separately using a transcription service or model. Here's a short video showing Gemini phrase endpointing in two scenarios. First, correctly handling pauses in natural conversation. Second, requesting a phone number (which is a common activity in a use case like customer support). You can see the Completeness check lines in the terminal output, printed each time Gemini processes a chunk of audio.

kwindla

64,190 Aufrufe • vor 1 Jahr

AI Sprawl is the silent productivity killer. Here's the story of how one engineer built the antidote. We were flipping through ChatGPT, Perplexity, and Gemini, copying and pasting context as well as uploading files from our ClickUp and other apps. Then I had a flashback to before ClickUp existed and realized it's the exact same problem we solved, but for app sprawl. It's freaking happening again, but this time with AI! I had a vivid dream where our 'One app to replace them all' billboards turned into 'One AI to rule them all'. We knew we were meant to solve this problem. Today we're launching Brain MAX. A fully-native desktop app that gives you productivity superpowers... ✨ Use all AI models, from one app. ✨ Connect all your apps to AI - not surface level, but deeply. ✨ Complete tasks, by giving them to AI. ✨ Schedule meetings, send messages, and emails. ✨ Daily agenda with prepped context for each of your meetings. ✨ Talk to Text (4x faster than typing) that writes like you, and even @ mentions names perfectly. This would take 5+ separate apps before today. Brain MAX was built in our innovation labs, where a small but mighty 100x team is cookin' around the clock. Our mission with MAX is to maximize human productivity. Reply "BRAIN" if you want access.
1:26

AI Sprawl is the silent productivity killer. Here's the story of how one engineer built the antidote. We were flipping through ChatGPT, Perplexity, and Gemini, copying and pasting context as well as uploading files from our ClickUp and other apps. Then I had a flashback to before ClickUp existed and realized it's the exact same problem we solved, but for app sprawl. It's freaking happening again, but this time with AI! I had a vivid dream where our 'One app to replace them all' billboards turned into 'One AI to rule them all'. We knew we were meant to solve this problem. Today we're launching Brain MAX. A fully-native desktop app that gives you productivity superpowers... ✨ Use all AI models, from one app. ✨ Connect all your apps to AI - not surface level, but deeply. ✨ Complete tasks, by giving them to AI. ✨ Schedule meetings, send messages, and emails. ✨ Daily agenda with prepped context for each of your meetings. ✨ Talk to Text (4x faster than typing) that writes like you, and even @ mentions names perfectly. This would take 5+ separate apps before today. Brain MAX was built in our innovation labs, where a small but mighty 100x team is cookin' around the clock. Our mission with MAX is to maximize human productivity. Reply "BRAIN" if you want access.

Zeb Evans

74,482 Aufrufe • vor 1 Jahr