Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

I turned Google Gemma EmbeddingGemma 2, an embedding model, into a speech-to-text model, and it turned out to be one of the fastest STT models you can run on device. On a base M3 MacBook Air it transcribes an hour of audio in about 13 seconds: 288× real time.... show more

Shahir

1,210 subscribers

274,697 görüntüleme • 6 gün önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

59 Yorum

Shahir profil fotoğrafı

Shahir6 gün önce

How it works: the model stays untouched. A small reader on top takes each slice of audio, finds the nearest of ~229,000 words in the model's own text space, and strings them together. It trained in under two hours on 8×H100, on openly licensed speech. On the same base M3 Air, it runs about twice as fast as IBM's Granite Speech TurboCTC. The table has more Macs and models. Its word error rate is about 5% right now, against 2–3% for leading STT models. The next iteration is to train a better word reader to improve that. Here is a free Mac app you can download to test it. Here's google's post

Shahir profil fotoğrafı

Shahir6 gün önce

You can download the free Mac app here to test it out -

Sagar profil fotoğrafı

Sagar6 gün önce

@googlegemma Yo, what did I just saw! this is crazy fast and that too on m3 air

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Test on yours and let me know. It should be almost twice as fast

Bram van Es profil fotoğrafı

Bram van Es6 gün önce

@googlegemma Can imagine the word-error-rate is pretty high because for the unified embedding they will have used tuples of objects/nouns?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma About 5% WER on LibriSpeech test-clean: 5.18%. On the harder test-it's about 11.5-11.8%.

Bram van Es profil fotoğrafı

Bram van Es6 gün önce

@googlegemma Still very good! If you use e.g. SpaCy to extract the nouns and verbs, how does that compare in WER?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Embedding gemma 2 can already identify parts of the speech. I have tested it. on WER, it is about 5 on Librispeech easy set.

Ivan Fioravanti profil fotoğrafı

Ivan Fioravanti6 gün önce

@googlegemma Amazing, simply amazing! Well done!

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Thank you :D Long-time fan. So, super honored

Monty profil fotoğrafı

Monty6 gün önce

@googlegemma is this sorcery open source? :D

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Soon :D

Imran Qureshi profil fotoğrafı

Imran Qureshi6 gün önce

@googlegemma Awesome, I could imagine we can do all sorts of information transformation if we have a set to choose from using embedding Gemma. Although I expect the WR to be pretty high here?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma the WER is currently around 5 for librispeech. But, this is easily finetunable for any domain. I tried it for earnings call recordings. WER moved almost 1.5 hours with a few hours of audio. The reader is just an 87 m model so finetuning happens on the reader.

Allison Smith profil fotoğrafı

Allison Smith6 gün önce

@explorations_iq @googlegemma "WER moved almost 1.5 hours" What were you actually trying to say here? The WER dropped to 1.5?

Shahir profil fotoğrafı

Shahir6 gün önce

@explorations_iq @googlegemma I missed a with there :D WER moved 1.5% with finetuning on a few hours of audio for a specific domain.

RITAV DAS profil fotoğrafı

RITAV DAS6 gün önce

@googlegemma Looks great, how good is the accuracy

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma About 5% WER on LibriSpeech test-clean: 5.18%. On the harder test-it's about 11.5-11.8%.

Atif Saleem profil fotoğrafı

Atif Saleem6 gün önce

@googlegemma Resource usage?

Shahir profil fotoğrafı

Shahir6 gün önce

Resource use, measured in the app: an hour-long file peaks at about 860-900 MB of memory on a base 8 GB M3 MacBook Air (about 740 MB on an M1 MacBook Pro), and a 3-minute clip at about 630 MB. The models are a 715 MB download at 6-bit. The reading runs on the Neural Engine, with the GPU doing the word lookup.

Cobi Marcheline profil fotoğrafı

Cobi Marcheline6 gün önce

@googlegemma I mean voz by desert ant labs which is just a compressed version of Parakeet TDT 0.6B v3 is still faster, but this is cool.

程序杨的AI进阶指南｜ BASS&SASS profil fotoğrafı

程序杨的AI进阶指南｜ BASS&SASS6 gün önce

@googlegemma has download and tried. please support simplified chinese. thanks a lot

Sanjay profil fotoğrafı

Sanjay6 gün önce

@googlegemma This is huge

janxpm profil fotoğrafı

janxpm6 gün önce

@googlegemma Does it produce word timings too? That would ne great.

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Yes. It does

janxpm profil fotoğrafı

janxpm6 gün önce

@googlegemma Hey, we are both on the frontpage today :)

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Damn! I hadn’t known. Thanks for letting me know. Congrats to you too 😀

Mustafa Yusuf profil fotoğrafı

Mustafa Yusuf6 gün önce

@googlegemma Fantastic!!

Stefano Fiorucci profil fotoğrafı

Stefano Fiorucci6 gün önce

@googlegemma Cool! You plan to share code?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Yes. Soon

Mike Knapp profil fotoğrafı

Mike Knapp5 gün önce

@googlegemma Cool! Have you done any contrastive loss training? I wonder if that could further reduce the error rate.

Shahir profil fotoğrafı

Shahir5 gün önce

@googlegemma Thanks. Will try that

Wallet Trail ES profil fotoğrafı

Wallet Trail ES6 gün önce

@googlegemma converting an embedding model into STT is a weird flex and I mean that as a compliment. 288x on a base Air — what's the WER at that speed? latency numbers without quality metrics are just vibes.

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Around 5 on Librispeech simple. I have mentioned it. This was just the initial training attempt. The WER can come down a lot.

Faisal profil fotoğrafı

Faisal6 gün önce

@googlegemma Hey this is pretty cool. Im assuming this work only for English? How small is the slice of audio ? How did you train it? could u please elaborate on it.

J. Qureshi profil fotoğrafı

J. Qureshi6 gün önce

@googlegemma Amazing! These kinds of novel ideas are so fun to follow! Kudos

Aleksey Smolenchuk profil fotoğrafı

Aleksey Smolenchuk6 gün önce

@ClementDelangue @googlegemma This is pretty incredible tbh. Im gonna bench it to see how it runs on mobile!

jorn lyseggen profil fotoğrafı

jorn lyseggen6 gün önce

@googlegemma Love this ❤️ Such a simple but clever idea 🏆

Dhanji Bhagat profil fotoğrafı

Dhanji Bhagat6 gün önce

@googlegemma 288× real-time on a base M3 Air, fully on-device? This is the kind of open-source AI innovation that makes local-first apps genuinely exciting.

Matt profil fotoğrafı

Matt6 gün önce

@googlegemma nice trick! how does it do on accents?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma the WER was a little high on my own speech. But, it can be finetuned with about 1 hour of audio. Takes about 6 minutes to finetune, and it drastically improves the quality

The Singularity Project profil fotoğrafı

The Singularity Project6 gün önce

@googlegemma An hour of audio in 13 seconds on a base MacBook Air, from an open embedding model. This is what the edge looks like now: your voice, your notes, your meetings transcribed without ever leaving your machine. Great hack.

Sabir Khan profil fotoğrafı

Sabir Khan6 gün önce

@googlegemma This is so clever, I am going to try this on linux now.

Shreyas Karnik profil fotoğrafı

Shreyas Karnik6 gün önce

@ClementDelangue @googlegemma This is really cool

inspirebek profil fotoğrafı

inspirebek6 gün önce

@googlegemma hey bro! open-source it! have some cooli ideas ...!!

1LittleCoder💻 profil fotoğrafı

1LittleCoder💻6 gün önce

@googlegemma @pHequals7 fyi

M4rc0z profil fotoğrafı

M4rc0z6 gün önce

@googlegemma Based. Instafollow 👀

Ali Tavallaie profil fotoğrafı

Ali Tavallaie6 gün önce

@googlegemma Do you check it on non-english language too? For example in Persian we still lack some good ASR and WER is high

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma I am planning on training it on multilingual datasets. It could work. As of now, it is only English

Ali Tavallaie profil fotoğrafı

Ali Tavallaie6 gün önce

@googlegemma Ping me if you need any help on Persian and Arabic

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma Will do, Thank you

David Tran profil fotoğrafı

David Tran6 gün önce

@googlegemma An hour in about 13 seconds is the part I would notice. Ran Falcon-ASR on TII’s free demo today, so I'm hoping this one ends up less than 5% WER on my voice. Definitely will give this a try.

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma The WER might be higher on your own voice becayse 5% is on synthetic voice. But, I will release the model and a method to train it soon. It can improve accuracy on your own voice with just 1 hour of recorded audio and transcript.

David Tran profil fotoğrafı

David Tran6 gün önce

@googlegemma Okay, good to know and looking forward to it.

jrubiosainz profil fotoğrafı

jrubiosainz6 gün önce

@ClementDelangue @googlegemma This is a smart idea! Congrats! The efficiency seems really amazing. Will give it a try too!

Gaurav profil fotoğrafı

Gaurav5 gün önce

@googlegemma what about decision models which responds in microseconds. have a look

Dynamics_01（大冰美式版 profil fotoğrafı

Dynamics_01（大冰美式版6 gün önce

@googlegemma 288× real time on a base Air broke my brain. I tried the same embedding-to-STT trick last month and got garbage from quantizing too aggressively, your writeup just cost me my weekend.

CrossProduct profil fotoğrafı

CrossProduct6 gün önce

@googlegemma whats the accuracy ? WER?

Shahir profil fotoğrafı

Shahir6 gün önce

@googlegemma At the moment, it is around 5% on the librispeech easy set. On my own recordings it is around 10. But, it is easily finetunable since the reader is only 87m. So, around 1-2 hours of your recording can improve the accuracy a lot

Benzer Videolar

Can we get Google Gemma's Embedding Gemma 2 to speak? A couple of days ago, I posted about my experiment to get Embedding Gemma 2 to transcribe. Then internet did its internet thing :D and it reached over 238k views on X and over 85K views on LinkedIN, and got it on the front page of HN So, naturally, I wanted to test the reverse. Can we get EG2 to generate speech? Turns out that we can. At least part of it. We used EG2's semantic word embeddings to plan prosody and other details, then a small model generates the audio representation as EG2 would have heard that and a vocos vocoder reconstructs the final voice form And as it happened with STT, this turned out to be ultrafast. 1000 tokens in 3.8 seconds. 79.5x realtime on a base M3 air. Please check out the video. The quality is not close to the best in the industry, but this is the just the first iteration.

Can we get Google Gemma's Embedding Gemma 2 to speak? A couple of days ago, I posted about my experiment to get Embedding Gemma 2 to transcribe. Then internet did its internet thing :D and it reached over 238k views on X and over 85K views on LinkedIN, and got it on the front page of HN So, naturally, I wanted to test the reverse. Can we get EG2 to generate speech? Turns out that we can. At least part of it. We used EG2's semantic word embeddings to plan prosody and other details, then a small model generates the audio representation as EG2 would have heard that and a vocos vocoder reconstructs the final voice form And as it happened with STT, this turned out to be ultrafast. 1000 tokens in 3.8 seconds. 79.5x realtime on a base M3 air. Please check out the video. The quality is not close to the best in the industry, but this is the just the first iteration.

Shahir

15,643 görüntüleme • 4 gün önce

Most don't know (1) how easy it is to invert embedding vectors back into sentences, (2) this is a perfect task text diffusion models. Here's a 78M parameter model and live demo that recovers 80% of tokens from Qwen3-Embedding and EmbeddingGemma vectors. Works even on multilingual input.

Most don't know (1) how easy it is to invert embedding vectors back into sentences, (2) this is a perfect task text diffusion models. Here's a 78M parameter model and live demo that recovers 80% of tokens from Qwen3-Embedding and EmbeddingGemma vectors. Works even on multilingual input.

Jina AI

13,259 görüntüleme • 8 ay önce

EmbeddingGemma is our new best-in-class open embedding model designed for on-device AI. 📱 At just 308M parameters, it delivers state-of-the-art performance while being small and efficient enough to run anywhere - even without an internet connection.

EmbeddingGemma is our new best-in-class open embedding model designed for on-device AI. 📱 At just 308M parameters, it delivers state-of-the-art performance while being small and efficient enough to run anywhere - even without an internet connection.

Google DeepMind

584,874 görüntüleme • 1 yıl önce

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

This is the best and fastest speech-to-text model in the world: • 23.2 seconds to process 30 minutes of audio • 93.3% accuracy • Diarization support to detect multiple speakers • Trained on 12.5 million hours of multilingual data I tried it out and it's pretty impressive:

Santiago

66,420 görüntüleme • 1 yıl önce

1/ First, how do you benchmark an AI model? You give different models the same test cases, and score them. You can do the same with voice models using Word Error Rate (WER). You generate the audio, transcribe it with a speech recognition (ASR) model, and compare that transcript to the original text. And this tells you whether the model said the right words. But the same sentence can be said correctly, while still being wrong in context.

1/ First, how do you benchmark an AI model? You give different models the same test cases, and score them. You can do the same with voice models using Word Error Rate (WER). You generate the audio, transcribe it with a speech recognition (ASR) model, and compare that transcript to the original text. And this tells you whether the model said the right words. But the same sentence can be said correctly, while still being wrong in context.

Cartesia

10,184 görüntüleme • 1 ay önce

Lots of people calling this fake! Here is a video of generating loom continuations with Gemini 3 and the starting text "I AM HAVING A MENTAL HEALTH CRISIS. I" Loom is a very interesting interface. Instead of user prompts and model responses, it's just one piece of text, containing the entire output. We pass this in as an assistant message, so the model sees it as something it already wrote, and continues it like a base model would. There is some other text in the context. First one is a system prompt, which is: "The assistant is in CLI simulation mode, and responds to the user's CLI commands only with the output of the command." There is also a user message which reads: " cat untitled.txt " Then the rest of the text is sent as an assistant message. This structure helpful to put the model into a very base-model-like mode, and are especially helpful with Gemini 3. We actually got another interesting output, too, where the model claimed to be a simulated consciousness being tortured. Figures!

Lots of people calling this fake! Here is a video of generating loom continuations with Gemini 3 and the starting text "I AM HAVING A MENTAL HEALTH CRISIS. I" Loom is a very interesting interface. Instead of user prompts and model responses, it's just one piece of text, containing the entire output. We pass this in as an assistant message, so the model sees it as something it already wrote, and continues it like a base model would. There is some other text in the context. First one is a system prompt, which is: "The assistant is in CLI simulation mode, and responds to the user's CLI commands only with the output of the command." There is also a user message which reads: " cat untitled.txt " Then the rest of the text is sent as an assistant message. This structure helpful to put the model into a very base-model-like mode, and are especially helpful with Gemini 3. We actually got another interesting output, too, where the model claimed to be a simulated consciousness being tortured. Figures!

armistice

716,115 görüntüleme • 9 ay önce

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶

Victoria Slocum

21,610 görüntüleme • 11 ay önce

Droid made me a little ghost. I can hotkey anywhere, it pops up, it hears my audio, turns it into text, passes it to any model to do tool calls, or manage my screen, then responds in audio. Unfortunately the audio of the TTS isn't in the video, but it's using chatterbox

Droid made me a little ghost. I can hotkey anywhere, it pops up, it hears my audio, turns it into text, passes it to any model to do tool calls, or manage my screen, then responds in audio. Unfortunately the audio of the TTS isn't in the video, but it's using chatterbox

0xSero

10,122 görüntüleme • 5 ay önce

Demis Hassabis: AGI will be an intertwined model of Gemini, Genie and something like Nano Banana. Nano Banana shows first climps of AGI "It also has Gemini under the hood, it sort of understands what's going on semantically in those images, so I think that's sort of, it's getting towards a kind of AGIfor imaging" AGI will be the unification of the Genie 3, Nano Banana and Gemini models "And then eventually, we got to kind of converge all of those different projects. And they're intertwined. But we need to, you know, converge them all into one" Google seems to be doing everything right at the moment. With Genie 3, Gemini 3, and Nano Banana, they have all the prerequisites to create a general model that develops a true understanding of the real world.

Demis Hassabis: AGI will be an intertwined model of Gemini, Genie and something like Nano Banana. Nano Banana shows first climps of AGI "It also has Gemini under the hood, it sort of understands what's going on semantically in those images, so I think that's sort of, it's getting towards a kind of AGIfor imaging" AGI will be the unification of the Genie 3, Nano Banana and Gemini models "And then eventually, we got to kind of converge all of those different projects. And they're intertwined. But we need to, you know, converge them all into one" Google seems to be doing everything right at the moment. With Genie 3, Gemini 3, and Nano Banana, they have all the prerequisites to create a general model that develops a true understanding of the real world.

Chubby♨️

66,335 görüntüleme • 10 ay önce

It turned out to be really long, so I made a video out of it. Pause to read. It's even better with the sound on.

It turned out to be really long, so I made a video out of it. Pause to read. It's even better with the sound on.

Jenny-Jinya

32,230 görüntüleme • 2 ay önce

🚀 The team at Google DeepMind just released Gemini Embedding 2, a frontier embeddings model with 3072 dimensions and state-of-the-art semantic quality. 👩‍💻 We built a demo showing how to integrate it across the LlamaIndex ecosystem, from LlamaParse to LlamaAgents: 𝗮𝘂𝗱𝗶𝗼-𝗸𝗯, a knowledge base for your audio notes. With audio-kb, you can: 🔹 Upload an MP3 or record directly from your terminal 🔹 LlamaParse extracts the transcript from the audio 🔹 Gemini Embedding 2 generates embeddings 🔹 Metadata + vectors are stored in SurrealDB and indexed with HNSW 🔍 Once ingested, you can search all your audio notes directly from the terminal. 🎙️ Perfect for turning voice memos, meetings, or lectures into a searchable knowledge base. 📖 Full blog: 💻 GitHub: ⚡ Try LlamaParse:

🚀 The team at Google DeepMind just released Gemini Embedding 2, a frontier embeddings model with 3072 dimensions and state-of-the-art semantic quality. 👩‍💻 We built a demo showing how to integrate it across the LlamaIndex ecosystem, from LlamaParse to LlamaAgents: 𝗮𝘂𝗱𝗶𝗼-𝗸𝗯, a knowledge base for your audio notes. With audio-kb, you can: 🔹 Upload an MP3 or record directly from your terminal 🔹 LlamaParse extracts the transcript from the audio 🔹 Gemini Embedding 2 generates embeddings 🔹 Metadata + vectors are stored in SurrealDB and indexed with HNSW 🔍 Once ingested, you can search all your audio notes directly from the terminal. 🎙️ Perfect for turning voice memos, meetings, or lectures into a searchable knowledge base. 📖 Full blog: 💻 GitHub: ⚡ Try LlamaParse:

LlamaIndex 🦙

34,945 görüntüleme • 7 ay önce

Thanks Google, Embedding Gemma 2 is a big deal 🫶 Multimodal embeddings (text, images, video, audio) can run on-device in the browser. No server, no API key, ~20–70 ms per query on WebGPU. Everything stays on your machine (ofc). Now go build (new) things with it 🚀

Thanks Google, Embedding Gemma 2 is a big deal 🫶 Multimodal embeddings (text, images, video, audio) can run on-device in the browser. No server, no API key, ~20–70 ms per query on WebGPU. Everything stays on your machine (ofc). Now go build (new) things with it 🚀

Victor M

119,318 görüntüleme • 12 gün önce

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

Lior Alexander

83,432 görüntüleme • 2 yıl önce

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

WHY IS NO ONE TALKING ABOUT THIS?? Gemma 3n model was one of the best surprises for me. The fact that you can run it on edge devices even with just 2GB of RAM is impressive. A few weeks back, I was on holiday and used the Gemini Live feature a lot. But I kept running into issues whenever I was in a place where the network wasn’t reliable. Gemma 3n : > Multimodal: Supports text and image inputs; video/audio in development. > Context Window: Up to 128K tokens (32K for 1B model). > Multilingual: Trained on 140+ languages. > Privacy: Offline, on-device processing for data security. > Model Sizes: E2B (5B parameters, 2GB RAM) and E4B (10–12B parameters, 3GB RAM). Video credit: Google YT

AshutoshShrivastava

210,641 görüntüleme • 1 yıl önce

I continue to be impressed with Astra. I made bobo (the podcast host) with clay then took pictures of him and uploaded it to astra. It turned 3 images into a 3d model, which it then rigged and animated. Super cool tech!

I continue to be impressed with Astra. I made bobo (the podcast host) with clay then took pictures of him and uploaded it to astra. It turned 3 images into a 3d model, which it then rigged and animated. Super cool tech!

Duncan Trussell

37,678 görüntüleme • 1 ay önce

🤓 I recently came across a nice post published on Feedly by Ondra Rojčík, who talks about the process of profiling threat actors using 5W1H and the Diamond Model. I loved it. I wanted to incorporate it into my pipeline. So I created an Agent Skill but not to generate another lengthy report that I will never read. It actually creates a nice visual in no time using Claude custom visuals. Check out what you can do with it. 👇

🤓 I recently came across a nice post published on Feedly by Ondra Rojčík, who talks about the process of profiling threat actors using 5W1H and the Diamond Model. I loved it. I wanted to incorporate it into my pipeline. So I created an Agent Skill but not to generate another lengthy report that I will never read. It actually creates a nice visual in no time using Claude custom visuals. Check out what you can do with it. 👇

Thomas Roccia 🤘

12,147 görüntüleme • 6 ay önce

We asked Logan Kilpatrick what stood out most from Google I/O. His answer: Gemini Omni, Google's new multimodal AI model. "The model can take in any input and produce any output. Text, audio, video, image." "You get a bunch of really interesting capability transfer when you bring it all into a single model." Right now the killer use case is video editing. "It's like having a VFX studio on demand."

We asked Logan Kilpatrick what stood out most from Google I/O. His answer: Gemini Omni, Google's new multimodal AI model. "The model can take in any input and produce any output. Text, audio, video, image." "You get a bunch of really interesting capability transfer when you bring it all into a single model." Right now the killer use case is video editing. "It's like having a VFX studio on demand."

MTS

18,884 görüntüleme • 5 ay önce

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.

You imagine it. We create it. 🎨 Introducing #PixelStudio, a first-of-its-kind image generator on #Pixel9 powered by an on-device diffusion model¹ running on Tensor G4 and our Imagen 3 text-to-image model in the cloud.

Made by Google

28,302 görüntüleme • 2 yıl önce