Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

does your audio-input LLM secretly know how to generate speech? yes but it sounds like a scary demon! deepdream but for speech: starting with noise audio, I tried optimizing the probability of a desired transcription, while keeping the model frozen. volume up! 🎧

Matt Henderson

80,686 subscribers

84,326 görüntüleme • 6 gün önce •via X (Twitter)

Bilim & Teknoloji Eğitim Müzik

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

cocktail peanut

14,745 görüntüleme • 1 yıl önce

Along with text, images, video and code, Gemini is able to process raw audio signal end-to-end. 🔊 It can listen to and understand speech, making it not only useful for transcription but a model that has a much more nuanced perception of its environment. ↓

Along with text, images, video and code, Gemini is able to process raw audio signal end-to-end. 🔊 It can listen to and understand speech, making it not only useful for transcription but a model that has a much more nuanced perception of its environment. ↓

Google DeepMind

140,179 görüntüleme • 2 yıl önce

I love using transcription when taking notes, but i’m not a huge fan of a lot of the apps that offer it. I also don’t like in-meeting note takers. So naturally I built a Raycast extension for transcribing notes. It transcribes both audio input and system audio and uses AI to summarise and copy to clipboard

I love using transcription when taking notes, but i’m not a huge fan of a lot of the apps that offer it. I also don’t like in-meeting note takers. So naturally I built a Raycast extension for transcribing notes. It transcribes both audio input and system audio and uses AI to summarise and copy to clipboard

Thomas Cullen

13,837 görüntüleme • 3 ay önce

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

OpenAI's S2S preview is polished but it still thinks in steps. Speech → text → model → text → speech. That's not how humans converse. Introducing Hydra. A native speech-to-speech model that doesn't wait for turn-taking, doesn't flatten emotion into text, and doesn't break when you interrupt it mid-sentence. Hydra reasons asynchronously, speaks and listens simultaneously, and preserves emotion because it never leaves the audio domain. It's still in beta, but the shift is obvious. If you want early access, the link is in the comments. Here's a preview of what that looks like -

Sudarshan Kamath

328,927 görüntüleme • 4 ay önce

This is not a joke! 🐬 Excited to share DolphinGemma the first audio-to-audio for dolphin communication! Yes, a model that predicts tokens on how dolphin speech! > DolphinGemma is the first LLM trained specifically to understand dolphin language patterns. > Leverages 40 years of data from Dr. Denise Herzing's unique collection > Works like text prediction, trying to "complete" dolphin whistles and sounds > Use wearable hardware (Google Pixel 9) to capture and analyze sounds in the field. > Dolphin Gemma is designed to be fine-tuned with new data > Weights coming soon! Research like this is why I love AI even more! ♥️

This is not a joke! 🐬 Excited to share DolphinGemma the first audio-to-audio for dolphin communication! Yes, a model that predicts tokens on how dolphin speech! > DolphinGemma is the first LLM trained specifically to understand dolphin language patterns. > Leverages 40 years of data from Dr. Denise Herzing's unique collection > Works like text prediction, trying to "complete" dolphin whistles and sounds > Use wearable hardware (Google Pixel 9) to capture and analyze sounds in the field. > Dolphin Gemma is designed to be fine-tuned with new data > Weights coming soon! Research like this is why I love AI even more! ♥️

Philipp Schmid

112,244 görüntüleme • 1 yıl önce

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

You can now generate real-time speech that sounds conversational. Microsoft just open-sourced VibeVoice, a real-time text-to-speech system with ~300 ms first audio latency and streaming input. It handles long conversations without falling apart. 𝗧𝗵𝗶𝘀 𝗺𝗼𝗱𝗲𝗹 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗹𝗼𝗻𝗴, 𝗺𝘂𝗹𝘁𝗶-𝘀𝗽𝗲𝗮𝗸𝗲𝗿 𝘀𝗽𝗲𝗲𝗰𝗵. It produces up to 90 minutes of audio. It supports up to four distinct speakers. Turn-taking stays consistent over long sessions. 𝗜𝘁 𝘄𝗼𝗿𝗸𝘀 𝗯𝘆 𝗿𝗲𝗱𝘂𝗰𝗶𝗻𝗴 𝘁𝗶𝗺𝗲 𝗿𝗲𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻. Audio compresses into semantic and acoustic tokens. They run at 7.5 Hz instead of frame-level audio. A language model predicts structure. A diffusion head restores acoustic detail. 𝗜𝘁 𝗮𝗹𝗹𝗼𝘄𝘀 𝗹𝗼𝘄-𝗹𝗮𝘁𝗲𝗻𝗰𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝗮𝘂𝗱𝗶𝗼. The real-time variant streams text incrementally. First speech arrives in ~300 ms. A WebSocket demo shows live generation. The code is MIT-licensed and research-only. The repo already passed 20k GitHub stars.

Lior Alexander

61,122 görüntüleme • 6 ay önce

Today we’re launching Stable Audio 2.5: The first audio model built for enterprise-grade sound production 🔊 Audio influences brand engagement by 86%, but few enterprises are leveraging audio as an extension of their brand, making customized sound an untapped differentiator. Stable Audio 2.5 is purpose-built for this opportunity to create customizable, high-quality audio at scale, with capabilities that include: ▶️ Improved musical composition: Generate full songs with multi-part structure, meaning a clear intro, middle, and outro. ▶️ Audio inpainting: Input audio, select where the track should start, and the model uses the context to generate the rest of the track. ▶️ Customization: Our team can fine-tune Stable Audio 2.5 to help enterprises create the right sound for their brand. ▶️ Faster inference: The model can generate up to three-minute long tracks in under two seconds on a GPU, outputting in just eight steps (compared to ~50 in the previous model). You can learn more here 👉

Stability AI

67,587 görüntüleme • 10 ay önce

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

AshutoshShrivastava

122,990 görüntüleme • 1 yıl önce

Elon Musk's Wisconsin Town Hall Q&A + Speech ✅ Improved audio for easier listening ✅ Silences removed to save you time Q&A at start, Elon's speech starts at 1h 25m. FYI, in this edit I put the Q&A *first* but at the event, the Q&A followed the speech. Separate edits are coming with just the speech (and) one with just the Q&A. Share them with someone you know!

Elon Musk's Wisconsin Town Hall Q&A + Speech ✅ Improved audio for easier listening ✅ Silences removed to save you time Q&A at start, Elon's speech starts at 1h 25m. FYI, in this edit I put the Q&A first but at the event, the Q&A followed the speech. Separate edits are coming with just the speech (and) one with just the Q&A. Share them with someone you know!

stevenmarkryan

1,017,453 görüntüleme • 1 yıl önce

How to Create Music That Matches Your Video's Sound with Suno Studio I made a quick video showing how I use Suno Studio to generate background music that fits the audio and mood of a video, similar to the example in the quoted post. This is the method I use at the moment but if you have a better approach, feel free to share it. Before starting, export the original audio from your video. I usually do this in CapCut. After generating a track you like in Suno Studio, you can import it back into CapCut and add it to your video. *This is not a paid partnership.

How to Create Music That Matches Your Video's Sound with Suno Studio I made a quick video showing how I use Suno Studio to generate background music that fits the audio and mood of a video, similar to the example in the quoted post. This is the method I use at the moment but if you have a better approach, feel free to share it. Before starting, export the original audio from your video. I usually do this in CapCut. After generating a track you like in Suno Studio, you can import it back into CapCut and add it to your video. *This is not a paid partnership.

Kōda

33,044 görüntüleme • 1 ay önce

Seed Audio 1.0 is now available on Runway for all paid plans. Generate up to 120 seconds of dynamic speech, sound design and music with simple text prompts. Try it now at the link below.

Seed Audio 1.0 is now available on Runway for all paid plans. Generate up to 120 seconds of dynamic speech, sound design and music with simple text prompts. Try it now at the link below.

Runway

31,268 görüntüleme • 28 gün önce

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,622 görüntüleme • 3 yıl önce

If you want some definitive proof that ABC destroyed the audio in Biden's interview, here you are. Left: original audio Right: denoised audio Bidens mic was EQ'd to pick up as much low end and high end ambient noise as possible, which is why his voice sounds so much more distorted than Stephanopoulos. Even with noise removal, the audio was beyond salvaging. You can listen for yourself. Biden team, if you're listening, never do another network interview/speech unless your own audio engineers get to set up the equipment.

If you want some definitive proof that ABC destroyed the audio in Biden's interview, here you are. Left: original audio Right: denoised audio Bidens mic was EQ'd to pick up as much low end and high end ambient noise as possible, which is why his voice sounds so much more distorted than Stephanopoulos. Even with noise removal, the audio was beyond salvaging. You can listen for yourself. Biden team, if you're listening, never do another network interview/speech unless your own audio engineers get to set up the equipment.

I Smoked Lindsey Graham

4,764,950 görüntüleme • 2 yıl önce

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 görüntüleme • 3 yıl önce

Droid made me a little ghost. I can hotkey anywhere, it pops up, it hears my audio, turns it into text, passes it to any model to do tool calls, or manage my screen, then responds in audio. Unfortunately the audio of the TTS isn't in the video, but it's using chatterbox

Droid made me a little ghost. I can hotkey anywhere, it pops up, it hears my audio, turns it into text, passes it to any model to do tool calls, or manage my screen, then responds in audio. Unfortunately the audio of the TTS isn't in the video, but it's using chatterbox

0xSero

10,122 görüntüleme • 2 ay önce

🇫🇷🇺🇸 BRIGITTE MACRON: "FREE SPEECH NEEDS LIMITS" "I do know that I am the United States, a country of free speech. But I think that liberty does have some rules. We have to be free but adapt to certain codes."

🇫🇷🇺🇸 BRIGITTE MACRON: "FREE SPEECH NEEDS LIMITS" "I do know that I am the United States, a country of free speech. But I think that liberty does have some rules. We have to be free but adapt to certain codes."

NewsForce

963,579 görüntüleme • 4 ay önce

Q: I enjoy listening to music with my headphones 🎧 How can I avoid hearing loss? A: 📶 Keep the volume below 60% of max on your device 🎧 Use well-fitted, noise-cancelling headphones. This can reduce the need to raise volume 🔊 Reduce time spent listening to loud sounds

Q: I enjoy listening to music with my headphones 🎧 How can I avoid hearing loss? A: 📶 Keep the volume below 60% of max on your device 🎧 Use well-fitted, noise-cancelling headphones. This can reduce the need to raise volume 🔊 Reduce time spent listening to loud sounds

World Health Organization (WHO)

53,711 görüntüleme • 1 yıl önce

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Today, we’re excited to introduce Miso One, the most emotive voice model in the world. Miso One is an 8-billion-parameter text-to-speech model for highly expressive speech generation. It emotes like a human and responds faster than a human, with just 110 milliseconds of latency. We’ve open-sourced the model weights, with API access coming soon. Hear how Miso One sounds in the thread below.

Aoden Teo

5,175,401 görüntüleme • 1 ay önce