Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Along with text, images, video and code, Gemini is able to process raw audio signal end-to-end. 🔊 It can listen to and understand speech, making it not only useful for transcription but a model that has a much more nuanced perception of its environment. ↓

Google DeepMind

1,486,615 subscribers

140,179 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Comments

Kekius-Maximus.com2 years ago

nice work

Adolfo Asorlin 👨🏼‍🚀2 years ago

This could lead to the development of more intelligent and helpful AI assistants, as well as more immersive and engaging experiences. I am eager to see what Gemini is capable of, and I am excited to see how it will shape the future of artificial intelligence. Thank you.

grate0x2 years ago

Very impressive work @GoogleDeepMind

Xi 🏴‍☠️2 years ago

we're soo back

UNRAVEL FUTURE2 years ago

Impressive thou!

Esthefany Pinella2 years ago

Y esto de los avances en la IA apenas comienza 🙌

BTB Finance2 years ago

Unbelievable

Dieter b/org2 years ago

you are Gemini?

🥷SoundNinja🥷2 years ago

Nice, thanks Google!

Joe Depp2 years ago

Cette innovation va engendrer une forte accélération dans le domaine de l'intelligence artificielle, comme on le constate par les usages exubérants dans cette vidéo. C'est une véritable prouesse d'être parvenu à un tel modèle eu égard au retard qu'avait google sur openai.

Related Videos

🚨 ELON: GROK WILL BE ABLE TO UNDERSTAND IMAGES, VIDEOS AND BEHAVIORS “It does, it actually does understand images at this point. It certainly is intended to work in sync. It's intended to be what's called a multimodal model, which means understanding text, images, video, and audio.” Peterson: “Okay, if it understands video, will it start to understand behavior?” Elon: “Yeah.” Source: DW

🚨 ELON: GROK WILL BE ABLE TO UNDERSTAND IMAGES, VIDEOS AND BEHAVIORS “It does, it actually does understand images at this point. It certainly is intended to work in sync. It's intended to be what's called a multimodal model, which means understanding text, images, video, and audio.” Peterson: “Okay, if it understands video, will it start to understand behavior?” Elon: “Yeah.” Source: DW

Mario Nawfal

58,182 views • 2 years ago

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

AshutoshShrivastava

122,990 views • 1 year ago

Gemini 3.1 Flash-Lite is the model to build always on multimodal AI Agents. I just built a memory agent with Google ADK that runs 24/7. Feed it text, images, audio, video or PDFs. It reads, connects and consolidates everything while you sleep. 100% Opensource code.

Gemini 3.1 Flash-Lite is the model to build always on multimodal AI Agents. I just built a memory agent with Google ADK that runs 24/7. Feed it text, images, audio, video or PDFs. It reads, connects and consolidates everything while you sleep. 100% Opensource code.

Shubham Saboo

24,930 views • 5 months ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

I just built an open NotebookLM clone! Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙️ The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works, by building it step-by-step! So by the end of this video, you'll learn how to: ↳ Process multimodal data (text, audio, video, URLs, and YouTube videos) into a format ready for LLMs ↳ Store everything in a vector database for fast retrieval ↳ Add a memory layer that remembers conversations and preferences for a personalized experience ↳ Chat with your knowledge base or generate podcasts using a fully open-source, locally running text-to-speech model The podcast generation feature is my favorite part! There's something powerful about turning written content into conversational audio that you can listen to while doing something else. The entire code is 100% open-source. I've shared a link in the replies! ____ Don't forget to drop a like if you enjoy my videos. It shows me I should be making more content like this. Cheers! :)

I just built an open NotebookLM clone! Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙️ The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works, by building it step-by-step! So by the end of this video, you'll learn how to: ↳ Process multimodal data (text, audio, video, URLs, and YouTube videos) into a format ready for LLMs ↳ Store everything in a vector database for fast retrieval ↳ Add a memory layer that remembers conversations and preferences for a personalized experience ↳ Chat with your knowledge base or generate podcasts using a fully open-source, locally running text-to-speech model The podcast generation feature is my favorite part! There's something powerful about turning written content into conversational audio that you can listen to while doing something else. The entire code is 100% open-source. I've shared a link in the replies! ____ Don't forget to drop a like if you enjoy my videos. It shows me I should be making more content like this. Cheers! :)

Akshay 🚀

83,187 views • 9 months ago

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

Lior Alexander

83,432 views • 2 years ago

I wanted to understand the proof and ended up making a video using opus 4.6. Still not completely happy with it, but maybe useful to others!

I wanted to understand the proof and ended up making a video using opus 4.6. Still not completely happy with it, but maybe useful to others!

Damek

15,727 views • 3 months ago

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations

Google AI

1,093,093 views • 7 months ago

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

RTFM can be seen as a learned renderer: it is an autoregressive diffusion transformer trained end-to-end on large-scale video data, and it learns to model 3D geometry, reflections, shadows and more just by observing them in its training set.

World Labs

15,870 views • 9 months ago

Share this for awareness. Watch till the end... there’s much more to a #bindi than just beauty. It represents tradition, identity, energy, and deep cultural meaning. Let’s not reduce it to a trend ..understand it, respect it, and share its true significance. ❤️

Share this for awareness. Watch till the end... there’s much more to a #bindi than just beauty. It represents tradition, identity, energy, and deep cultural meaning. Let’s not reduce it to a trend ..understand it, respect it, and share its true significance. ❤️

Rupsy Saini

12,961 views • 3 months ago

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

Sudarshan Kamath

328,927 views • 5 months ago

We’re excited to announce 𝗚𝗲𝗺𝗶𝗻𝗶: Google’s largest and most capable AI model. Built to be natively multimodal, it can understand and operate across text, code, audio, image and video - and achieves state-of-the-art performance across many tasks. 🧵

We’re excited to announce 𝗚𝗲𝗺𝗶𝗻𝗶: Google’s largest and most capable AI model. Built to be natively multimodal, it can understand and operate across text, code, audio, image and video - and achieves state-of-the-art performance across many tasks. 🧵

Google DeepMind

1,315,763 views • 2 years ago

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).

Happy GPT-5 Day to all who celebrate. Here's what I expect: It will be agentic-first (this is all but confirmed by OpenAI) It will be heavily multimodal (audio, video, music, tool use, and more) It will be a reasoning-first model as well (math, planning, etc) Directionally, I suspect we're looking at moving towards a convergence of "everything to everything" end-to-end models. Sort of like how Tesla improved FSD by doing neural networks for end-to-end perception-planning-control. Likewise, I suspect that, for the foreseeable future on general purpose models, we're going to converge towards "all in one" models. They will just tend to get smarter and smarter the more modalities we stuff into them (world models, math, image, text, video, audio, etc).

David Shapiro (L/0)

19,305 views • 11 months ago

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡

Vaibhav (VB) Srivastav

45,409 views • 1 year ago

COACHES SOLVING THEIR TEAMS PUZZLE As fans, trainers, parents, players we underestimate how much of a process it is for coaches to figure out their teams puzzle. There are so many variables that go in to a team having success and the only way to get to that end goal is by losing and making mistakes along the way. No team is perfect, no coach is perfect, but give me a coach that is constantly fighting for their team and I’ll rock with them all day!!

COACHES SOLVING THEIR TEAMS PUZZLE As fans, trainers, parents, players we underestimate how much of a process it is for coaches to figure out their teams puzzle. There are so many variables that go in to a team having success and the only way to get to that end goal is by losing and making mistakes along the way. No team is perfect, no coach is perfect, but give me a coach that is constantly fighting for their team and I’ll rock with them all day!!

Steve Dagostino

12,162 views • 1 year ago

Sonilo has released Sound Effects 1.0, a video-native audio model that can read on-screen motion, scene context, and timing. > It can make sound effects synced to the footage. > Video-to-Sound-Effects runs automatically without requiring a prompt. > Text-to-Sound-Effects is able to generate standalone assets from a written description.

Sonilo has released Sound Effects 1.0, a video-native audio model that can read on-screen motion, scene context, and timing. > It can make sound effects synced to the footage. > Video-to-Sound-Effects runs automatically without requiring a prompt. > Text-to-Sound-Effects is able to generate standalone assets from a written description.

🚨 AI News | TestingCatalog

15,523 views • 8 days ago

Gemini 3.1 Flash TTS is our most controllable text-to-speech model yet. With new Audio Tags, you can easily direct vocal style, delivery, and pace through text commands. 🧵

Gemini 3.1 Flash TTS is our most controllable text-to-speech model yet. With new Audio Tags, you can easily direct vocal style, delivery, and pace through text commands. 🧵

Google DeepMind

469,947 views • 3 months ago

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

AI at Meta

1,251,082 views • 7 months ago

Introducing OpenVoice 🔊 Instant voice cloning that requires only a short audio clip to replicate voice and generate speech in multiple languages and tones Listen to the examples They are able to replicate voices with only one sentence Demo to try below:

Introducing OpenVoice 🔊 Instant voice cloning that requires only a short audio clip to replicate voice and generate speech in multiple languages and tones Listen to the examples They are able to replicate voices with only one sentence Demo to try below:

Allen T.

32,935 views • 2 years ago

Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!

Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!

Ilaria Manco

44,994 views • 1 month ago