Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Next gen AI audio separation is here 🤯 AudioSep is a model that can separate audio events, musical instruments, and even enhance speech with natural language queries which makes this a versatile tool for different audio tasks.

Dreaming Tulpa 🥓👑

54,784 subscribers

126,038 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Comments

Dreaming Tulpa 🥓👑2 years ago

If you like Tweets like this, you might enjoy my weekly newsletter, #aiartweekly. A free, once–weekly e-mail round-up of the latest AI art news, interviews with artists and useful tools & resources. Join 2500+ subscribers here:

Sylvain Filoni2 years ago

And you can test AudioSep on @huggingface 🤗

Boise 🍄 Midtrip Art2 years ago

Ooof this is crazy Seems AI is mostly talked about when it comes to visual creation, but soon it will be even more prominent in music creation

Dreaming Tulpa 🥓👑2 years ago

Because it’s something we can grasp instantly when we look at it. Listening takes time. But like with images, AI enhanced or generated music will be everywhere in the next 2-5 years.

Coopdville.eth (Coop D'Ville)2 years ago

This is a game changer for my upcoming music releases that use samples... 🔥🔥🔥🎶🎶🎶🎹🎹🎹

Kickiniteasy2 years ago

so rad this tech in plugins for music production is very useful for isolating tracks historically meh... but with the new AI isolation in plugins 🤯 made a song last week using AI track isolation & our minds were blown in the studio opens up whole new worlds for sampling

Dreaming Tulpa 🥓👑2 years ago

Got an example? Would love to give it a listen!

Brennan Erbz2 years ago

I wonder why Bytedance is working on audio separation 🤔

CMO3332 years ago

Wow

M.A. Buth2 years ago

Sound cool!

Related Videos

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time. It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time. It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

Greg Brockman

4,360,961 views • 2 years ago

Today we launched Gemini 3.1 Flash TTS, our most expressive and controllable text-to-speech model yet. This launch [excitement] includes audio tags! 🗣🏷 Audio tags [explanatory] are a seamless way to guide vocal style, pace, and delivery using natural language commands embedded directly in your text. Want a different tempo or tone? [amazement] Just tag the audio to steer the AI-speech output! The model supports 70+ languages (24 of which are high-quality evaluated languages, including: Japanese, Hindi, and Arabic). Watch the audio tags in action in the demo below ↓

Today we launched Gemini 3.1 Flash TTS, our most expressive and controllable text-to-speech model yet. This launch [excitement] includes audio tags! 🗣🏷 Audio tags [explanatory] are a seamless way to guide vocal style, pace, and delivery using natural language commands embedded directly in your text. Want a different tempo or tone? [amazement] Just tag the audio to steer the AI-speech output! The model supports 70+ languages (24 of which are high-quality evaluated languages, including: Japanese, Hindi, and Arabic). Watch the audio tags in action in the demo below ↓

Google AI

202,949 views • 3 months ago

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

Ant Ling

21,365 views • 10 months ago

OSINT Tool: Extract Sounds from Audio Files 🎵🔍 Most audio tools just let you play or cut files. Finding a specific sound inside a long recording can be tedious. AudioGhost AI changes that. It uses a memory-optimized SAM-Audio model to extract specific sounds from audio files using text queries. You can: 🎯 Search for particular sounds in any audio 🖥️ Use a modern, user-friendly interface 💾 Work efficiently even with large files This is perfect for investigations, research, or any project where audio analysis matters. 🔗 Website link: __________ P.S. ♻️ Repost if you found this helpful. If you liked this post and would like to learn more methods and techniques to discover information about people, check out my OSINT Mastery course

OSINT Tool: Extract Sounds from Audio Files 🎵🔍 Most audio tools just let you play or cut files. Finding a specific sound inside a long recording can be tedious. AudioGhost AI changes that. It uses a memory-optimized SAM-Audio model to extract specific sounds from audio files using text queries. You can: 🎯 Search for particular sounds in any audio 🖥️ Use a modern, user-friendly interface 💾 Work efficiently even with large files This is perfect for investigations, research, or any project where audio analysis matters. 🔗 Website link: __________ P.S. ♻️ Repost if you found this helpful. If you liked this post and would like to learn more methods and techniques to discover information about people, check out my OSINT Mastery course

CyberSudo

16,674 views • 6 months ago

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

OK, this is insane.. Alibaba just dropped 4 new AI models at Apsara 2025, and they’re wild: → a 1 trillion parameter LLM → a vision model that codes from images → an omni-model for text/audio/video → and a new Wan 2.5 preview for video + audio gen more details below:👇

Hamza Khalid

26,707 views • 10 months ago

Introducing Gemini 3.1 Flash TTS 🗣️, our latest text to speech model with scene direction, speaker level specificity, audio tags, more natural + expressive voices, and support for 70 different languages. Available via our new audio playground in AI Studio and in the Gemini API!

Introducing Gemini 3.1 Flash TTS 🗣️, our latest text to speech model with scene direction, speaker level specificity, audio tags, more natural + expressive voices, and support for 70 different languages. Available via our new audio playground in AI Studio and in the Gemini API!

Logan Kilpatrick

803,494 views • 3 months ago

Suno Studio is absolutely incredible. AI Digital Audio Workstation (DAWs) are here, and we are going to create some incredible music with this. I love how you can generate each instrument with a text prompt. You can even record your own instruments too!

Suno Studio is absolutely incredible. AI Digital Audio Workstation (DAWs) are here, and we are going to create some incredible music with this. I love how you can generate each instrument with a text prompt. You can even record your own instruments too!

Jerrod Lew

87,295 views • 10 months ago

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Rohan Paul

96,354 views • 1 year ago

Today we’re launching Stable Audio 2.5: The first audio model built for enterprise-grade sound production 🔊 Audio influences brand engagement by 86%, but few enterprises are leveraging audio as an extension of their brand, making customized sound an untapped differentiator. Stable Audio 2.5 is purpose-built for this opportunity to create customizable, high-quality audio at scale, with capabilities that include: ▶️ Improved musical composition: Generate full songs with multi-part structure, meaning a clear intro, middle, and outro. ▶️ Audio inpainting: Input audio, select where the track should start, and the model uses the context to generate the rest of the track. ▶️ Customization: Our team can fine-tune Stable Audio 2.5 to help enterprises create the right sound for their brand. ▶️ Faster inference: The model can generate up to three-minute long tracks in under two seconds on a GPU, outputting in just eight steps (compared to ~50 in the previous model). You can learn more here 👉

Stability AI

67,618 views • 10 months ago

ElevenLabs v3 is now on Poe! With new audio tags, multi-speaker support, and options for over 70 languages, this category leading model gives you more control to create audio that feels more real, natural and expressive. (1/2)

ElevenLabs v3 is now on Poe! With new audio tags, multi-speaker support, and options for over 70 languages, this category leading model gives you more control to create audio that feels more real, natural and expressive. (1/2)

Poe

13,059 views • 11 months ago

Today we're releasing our first open source TTS model, TADA! TADA (Text Audio Dual Alignment) is a speech-language model that generates text and audio in one synchronized stream to reduce token-level hallucinations and improve latency. This means: → Zero content hallucinations across 1,000+ test samples → 5x faster than similar-grade LLM-based TTS → Fits much longer audio: 2,048 tokens cover ~700 seconds with TADA vs. ~70 seconds in conventional systems → Free transcript alongside audio with no added latency

Today we're releasing our first open source TTS model, TADA! TADA (Text Audio Dual Alignment) is a speech-language model that generates text and audio in one synchronized stream to reduce token-level hallucinations and improve latency. This means: → Zero content hallucinations across 1,000+ test samples → 5x faster than similar-grade LLM-based TTS → Fits much longer audio: 2,048 tokens cover ~700 seconds with TADA vs. ~70 seconds in conventional systems → Free transcript alongside audio with no added latency

Hume AI

270,568 views • 4 months ago

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,624 views • 3 years ago

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

Sophia Yang, Ph.D.

49,801 views • 1 year ago

Introducing Talk Arena🔊: the platform for interactively evaluating Large Audio Models! With the release of speech AI models like GPT4o, Gemini, Qwen-Audio, etc. which is the best? Cast your votes and help us decide🔥

Introducing Talk Arena🔊: the platform for interactively evaluating Large Audio Models! With the release of speech AI models like GPT4o, Gemini, Qwen-Audio, etc. which is the best? Cast your votes and help us decide🔥

Michael Ryan

17,425 views • 1 year ago

It's incredible that there's no live footage w/ audio of pre-reunion Velvets. This new fan-made vid does a great job of simulating it, with clear audio from a separate performance that lines up well. (That they're playing outside on a sunny day makes this even more interesting.)

It's incredible that there's no live footage w/ audio of pre-reunion Velvets. This new fan-made vid does a great job of simulating it, with clear audio from a separate performance that lines up well. (That they're playing outside on a sunny day makes this even more interesting.)

𝗗𝗘𝗔𝗗𝗦𝗢𝗨𝗡𝗗 💀

32,769 views • 4 months ago

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

Anabiya

56,404 views • 29 days ago

Hedra’s Character-3 is a game-changer for AI storytelling! 🔥✨ Upload an image, add speech and movement prompts, or even drop in an audio file, and it generates the full video for you. The Process is super easy: Image: Freepik Audio: elevenlabs (you can record yourself) Video: Hedra Video prompt: female reporter excitedly reporting the news, holds a microphone, wearing a professional suit, delivering a live broadcast., natural movements You can also use Hedra to generate audio and images! All in one 🚀🔥 What do you think of the result!?

Hedra’s Character-3 is a game-changer for AI storytelling! 🔥✨ Upload an image, add speech and movement prompts, or even drop in an audio file, and it generates the full video for you. The Process is super easy: Image: Freepik Audio: elevenlabs (you can record yourself) Video: Hedra Video prompt: female reporter excitedly reporting the news, holds a microphone, wearing a professional suit, delivering a live broadcast., natural movements You can also use Hedra to generate audio and images! All in one 🚀🔥 What do you think of the result!?

Amira Zairi

76,116 views • 1 year ago

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 views • 3 years ago

Gemini 3.1 Flash Live just dropped and it's available with LiveKit today. This is the first Gemini 3 native audio model on the Live API. Better instruction following, improved tool calling, reduced speaker drift, and support for 70+ languages. Audio in, audio out. No text conversion in between.

Gemini 3.1 Flash Live just dropped and it's available with LiveKit today. This is the first Gemini 3 native audio model on the Live API. Better instruction following, improved tool calling, reduced speaker drift, and support for 70+ languages. Audio in, audio out. No text conversion in between.

LiveKit

40,277 views • 4 months ago

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

AshutoshShrivastava

122,990 views • 1 year ago