Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with... show more

Meta Newsroom

259,330 subscribers

204,498 views • 7 months ago •via X (Twitter)

Arts Music Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

AI at Meta

1,250,414 views • 7 months ago

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

Anabiya

56,072 views • 11 days ago

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

ElevenLabs

450,763 views • 2 years ago

As part of Meta Movie Gen, we trained a 13B parameter audio generation model that can take a video + optional text prompts to generate high quality audio — including ambient sound, foley & instrumental background music — all synced to the video. Details ➡️

As part of Meta Movie Gen, we trained a 13B parameter audio generation model that can take a video + optional text prompts to generate high quality audio — including ambient sound, foley & instrumental background music — all synced to the video. Details ➡️

AI at Meta

76,692 views • 1 year ago

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,608 views • 3 years ago

Video to Audio is now here in #DreamMachine. To generate sound for your video generations, just select the new "Audio" button. Create with a single click or describe with prompts for more customized direction. Audio is available now in beta for free to all users.

Video to Audio is now here in #DreamMachine. To generate sound for your video generations, just select the new "Audio" button. Create with a single click or describe with prompts for more customized direction. Audio is available now in beta for free to all users.

Luma

2,260,160 views • 1 year ago

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Rohan Paul

96,354 views • 1 year ago

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,264,759 views • 1 year ago

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan

52,119 views • 9 months ago

Introducing the open-source Next.js Audio Starter Kit Add Text to Speech, Speech to Text, Sound Effects and Conversational AI to your product in minutes. Built with: • ElevenLabs SDK • Next.js + shadcn/ui • Tailwind CSS v4 Repo link in the comments below:

Introducing the open-source Next.js Audio Starter Kit Add Text to Speech, Speech to Text, Sound Effects and Conversational AI to your product in minutes. Built with: • ElevenLabs SDK • Next.js + shadcn/ui • Tailwind CSS v4 Repo link in the comments below:

ElevenLabs Developers

145,836 views • 1 year ago

Audio tools are ready on BudgetPixel AI。 Choose from 300+ preset tones, convert voice to text, or clone a voice for custom audio. From voiceovers and narration to ads, characters, and social videos, create the sound your content needs faster.

Audio tools are ready on BudgetPixel AI。 Choose from 300+ preset tones, convert voice to text, or clone a voice for custom audio. From voiceovers and narration to ads, characters, and social videos, create the sound your content needs faster.

BudgetPixel AI

69,753 views • 17 days ago

Kling VIDEO 2.6 — See the Sound, Hear the Visual "What is your most unforgettable moment from this year?" For us, it's definitely this Omni Launch Week with O1 and 2.6 now available! Share your most unforgettable moment using Kling 2.6, with natural voice, sound effects, and ambient sounds perfectly aligned with the visuals! All visuals and audio in this video were generated with the Kling 2.6 model.

Kling VIDEO 2.6 — See the Sound, Hear the Visual "What is your most unforgettable moment from this year?" For us, it's definitely this Omni Launch Week with O1 and 2.6 now available! Share your most unforgettable moment using Kling 2.6, with natural voice, sound effects, and ambient sounds perfectly aligned with the visuals! All visuals and audio in this video were generated with the Kling 2.6 model.

Kling AI

81,635 views • 7 months ago

Introducing Audio-to-Video in LTX. The third paradigm in AI video generation. Start from audio to get consistent voices, control over performance, and actions timed to the beat. We’re proud to partner with for this launch. Here’s how it works (sound on) 🧵

Introducing Audio-to-Video in LTX. The third paradigm in AI video generation. Start from audio to get consistent voices, control over performance, and actions timed to the beat. We’re proud to partner with for this launch. Here’s how it works (sound on) 🧵

LTX.io

306,393 views • 5 months ago

Check out the 4 new Google Photos features coming first to Pixel 8 and 8 Pro ↓ Whether it’s noise from wind, traffic, or barking dogs, Audio Magic Eraser in Google Photos reduces distracting sounds in your video in just a few taps! 🪄

Check out the 4 new Google Photos features coming first to Pixel 8 and 8 Pro ↓ Whether it’s noise from wind, traffic, or barking dogs, Audio Magic Eraser in Google Photos reduces distracting sounds in your video in just a few taps! 🪄

Google Photos

107,621 views • 2 years ago

You’ll want sound on for this. Announcing Lipdub Playground. With just a click, add a voice to your videos. Upload a video, type in a script (or drop in your own audio), and let your videos do the talking.

You’ll want sound on for this. Announcing Lipdub Playground. With just a click, add a voice to your videos. Upload a video, type in a script (or drop in your own audio), and let your videos do the talking.

Mirage

30,897 views • 1 year ago

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

Sophia Yang, Ph.D.

49,801 views • 1 year ago

Quick tip for anyone generating videos with Hailuo 2 or Midjourney Video since they don't generate with any sound. You can generate sound effects for free using MMAUDIO via huggingface. I'm sure a lot of you are familiar with it. It's video to audio synthesis and it's FREE! - Go to this link - Upload your video - Type in a description for the sound - Choose a duration - Click on submit Your video now has sound 👍

Quick tip for anyone generating videos with Hailuo 2 or Midjourney Video since they don't generate with any sound. You can generate sound effects for free using MMAUDIO via huggingface. I'm sure a lot of you are familiar with it. It's video to audio synthesis and it's FREE! - Go to this link - Upload your video - Type in a description for the sound - Choose a duration - Click on submit Your video now has sound 👍

Travis Davids

30,113 views • 1 year ago

Introducing Large Visual Memory Model (LVMM) 2.0, in partnership with Qualcomm Advanced visual AI is now on your phone, and in your home cameras. AI that understands your photos and videos, runs 100% privately and securely. ALL on-device. The beginning of on-device multimodal AI.

Introducing Large Visual Memory Model (LVMM) 2.0, in partnership with Qualcomm Advanced visual AI is now on your phone, and in your home cameras. AI that understands your photos and videos, runs 100% privately and securely. ALL on-device. The beginning of on-device multimodal AI.

Shawn Shen

1,204,535 views • 8 months ago

Today we launched Gemini 3.1 Flash TTS, our most expressive and controllable text-to-speech model yet. This launch [excitement] includes audio tags! 🗣🏷 Audio tags [explanatory] are a seamless way to guide vocal style, pace, and delivery using natural language commands embedded directly in your text. Want a different tempo or tone? [amazement] Just tag the audio to steer the AI-speech output! The model supports 70+ languages (24 of which are high-quality evaluated languages, including: Japanese, Hindi, and Arabic). Watch the audio tags in action in the demo below ↓

Today we launched Gemini 3.1 Flash TTS, our most expressive and controllable text-to-speech model yet. This launch [excitement] includes audio tags! 🗣🏷 Audio tags [explanatory] are a seamless way to guide vocal style, pace, and delivery using natural language commands embedded directly in your text. Want a different tempo or tone? [amazement] Just tag the audio to steer the AI-speech output! The model supports 70+ languages (24 of which are high-quality evaluated languages, including: Japanese, Hindi, and Arabic). Watch the audio tags in action in the demo below ↓

Google AI

202,556 views • 3 months ago