Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and... show more

AI at Meta

825,051 subscribers

1,250,498 Aufrufe • vor 7 Monaten •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Meta Newsroom

204,530 Aufrufe • vor 7 Monaten

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,608 Aufrufe • vor 3 Jahren

OSINT Tool: Extract Sounds from Audio Files 🎵🔍 Most audio tools just let you play or cut files. Finding a specific sound inside a long recording can be tedious. AudioGhost AI changes that. It uses a memory-optimized SAM-Audio model to extract specific sounds from audio files using text queries. You can: 🎯 Search for particular sounds in any audio 🖥️ Use a modern, user-friendly interface 💾 Work efficiently even with large files This is perfect for investigations, research, or any project where audio analysis matters. 🔗 Website link: __________ P.S. ♻️ Repost if you found this helpful. If you liked this post and would like to learn more methods and techniques to discover information about people, check out my OSINT Mastery course

OSINT Tool: Extract Sounds from Audio Files 🎵🔍 Most audio tools just let you play or cut files. Finding a specific sound inside a long recording can be tedious. AudioGhost AI changes that. It uses a memory-optimized SAM-Audio model to extract specific sounds from audio files using text queries. You can: 🎯 Search for particular sounds in any audio 🖥️ Use a modern, user-friendly interface 💾 Work efficiently even with large files This is perfect for investigations, research, or any project where audio analysis matters. 🔗 Website link: __________ P.S. ♻️ Repost if you found this helpful. If you liked this post and would like to learn more methods and techniques to discover information about people, check out my OSINT Mastery course

CyberSudo

16,095 Aufrufe • vor 5 Monaten

Meet SAM 3, a unified model that enables detection, segmentation, and tracking of objects across images and videos. SAM 3 introduces some of our most highly requested features like text and exemplar prompts to segment all objects of a target category. Learnings from SAM 3 will help power new features in Instagram Edits and Vibes, bringing advanced segmentation capabilities directly to creators. 🔗 Learn more:

Meet SAM 3, a unified model that enables detection, segmentation, and tracking of objects across images and videos. SAM 3 introduces some of our most highly requested features like text and exemplar prompts to segment all objects of a target category. Learnings from SAM 3 will help power new features in Instagram Edits and Vibes, bringing advanced segmentation capabilities directly to creators. 🔗 Learn more:

AI at Meta

190,702 Aufrufe • vor 8 Monaten

As detailed in the Meta Movie Gen technical report, today we’re open sourcing Movie Gen Bench: two new media generation benchmarks that we hope will help to enable the AI research community to progress work on more capable audio and video generation models. Movie Gen Video Bench is the largest and most comprehensive benchmark ever released for evaluating text-to-video generation. It includes a collection of 1,000+ prompts that cover concepts ranging from detailed human activity to animals, physics, unusual subjects and more — with broad coverage across different motion levels. Movie Gen Audio Bench is a first-of-its-kind benchmark aimed at evaluating video-to-audio and (text+video)-to-audio generation. It includes 527 generated videos and associated sound effects and music prompts covering a diverse set of ambient environments and sound effects. To enable fair and easy comparison to our models for future works, these new benchmarks include non cherry-picked generated videos and audio from Movie Gen. In releasing these new benchmarks we hope to promote fair & extensive evaluations in media generation research to enable greater progress in this field.

As detailed in the Meta Movie Gen technical report, today we’re open sourcing Movie Gen Bench: two new media generation benchmarks that we hope will help to enable the AI research community to progress work on more capable audio and video generation models. Movie Gen Video Bench is the largest and most comprehensive benchmark ever released for evaluating text-to-video generation. It includes a collection of 1,000+ prompts that cover concepts ranging from detailed human activity to animals, physics, unusual subjects and more — with broad coverage across different motion levels. Movie Gen Audio Bench is a first-of-its-kind benchmark aimed at evaluating video-to-audio and (text+video)-to-audio generation. It includes 527 generated videos and associated sound effects and music prompts covering a diverse set of ambient environments and sound effects. To enable fair and easy comparison to our models for future works, these new benchmarks include non cherry-picked generated videos and audio from Movie Gen. In releasing these new benchmarks we hope to promote fair & extensive evaluations in media generation research to enable greater progress in this field.

AI at Meta

156,273 Aufrufe • vor 1 Jahr

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,264,864 Aufrufe • vor 1 Jahr

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉

kyutai

47,949 Aufrufe • vor 1 Jahr

Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

AI at Meta

1,088,498 Aufrufe • vor 8 Monaten

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Rohan Paul

96,354 Aufrufe • vor 1 Jahr

Today we’re launching Stable Audio 2.5: The first audio model built for enterprise-grade sound production 🔊 Audio influences brand engagement by 86%, but few enterprises are leveraging audio as an extension of their brand, making customized sound an untapped differentiator. Stable Audio 2.5 is purpose-built for this opportunity to create customizable, high-quality audio at scale, with capabilities that include: ▶️ Improved musical composition: Generate full songs with multi-part structure, meaning a clear intro, middle, and outro. ▶️ Audio inpainting: Input audio, select where the track should start, and the model uses the context to generate the rest of the track. ▶️ Customization: Our team can fine-tune Stable Audio 2.5 to help enterprises create the right sound for their brand. ▶️ Faster inference: The model can generate up to three-minute long tracks in under two seconds on a GPU, outputting in just eight steps (compared to ~50 in the previous model). You can learn more here 👉

Stability AI

67,552 Aufrufe • vor 10 Monaten

Along with text, images, video and code, Gemini is able to process raw audio signal end-to-end. 🔊 It can listen to and understand speech, making it not only useful for transcription but a model that has a much more nuanced perception of its environment. ↓

Along with text, images, video and code, Gemini is able to process raw audio signal end-to-end. 🔊 It can listen to and understand speech, making it not only useful for transcription but a model that has a much more nuanced perception of its environment. ↓

Google DeepMind

140,169 Aufrufe • vor 2 Jahren

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Tencent Hy

122,706 Aufrufe • vor 10 Monaten

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

I stopped writing giant video prompts. Now I build the audio first with Seed Audio 1.0. Dialogue, pauses, emotion, sound cues, all in one generation. Then I feed that audio into Seedance and the video follows the rhythm on its own. Timing, pacing, even camera feeling, all coming from the sound. Audio first. Video second. This is a different way to make AI films.

Anabiya

56,072 Aufrufe • vor 13 Tagen

Introducing 𝘾𝙡𝙞𝙥𝘼𝙣𝙮𝙩𝙝𝙞𝙣𝙜, the first-ever multimodal AI clipping that lets you clip any moment from any video using visual, audio, and sentiment cues. 🔥 Analyze everything in any video ClipAnything uses state-of-the-art video understanding that analyzes each frame through visual, audio, and sentiment cues, identifying objects, scenes, actions, sounds, emotions, texts, and more. ⭐️ Use natural language prompts to find any moments You can use natural language prompts to find any scene, action, character, event, emotional moment, viral topic, and more. 🌶️ Create viral clips from any video Extending beyond talking-head videos, ClipAnything can clip any type of video, such as vlogs, sports, TV shows, news, music & videos with little to no dialogue. To learn more about it, please go to ClipAnything webpage:

Introducing 𝘾𝙡𝙞𝙥𝘼𝙣𝙮𝙩𝙝𝙞𝙣𝙜, the first-ever multimodal AI clipping that lets you clip any moment from any video using visual, audio, and sentiment cues. 🔥 Analyze everything in any video ClipAnything uses state-of-the-art video understanding that analyzes each frame through visual, audio, and sentiment cues, identifying objects, scenes, actions, sounds, emotions, texts, and more. ⭐️ Use natural language prompts to find any moments You can use natural language prompts to find any scene, action, character, event, emotional moment, viral topic, and more. 🌶️ Create viral clips from any video Extending beyond talking-head videos, ClipAnything can clip any type of video, such as vlogs, sports, TV shows, news, music & videos with little to no dialogue. To learn more about it, please go to ClipAnything webpage:

OpusClip

74,286 Aufrufe • vor 1 Jahr

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

How to train a model that actually understands both audio and text like Voxtral from Mistral AI? Here is a quick video walkthrough of the paper.

Sophia Yang, Ph.D.

49,801 Aufrufe • vor 1 Jahr

experiment with bytedance's new SEED audio model... it's tricky to get consistent audio out of seedance prompts, esp if you want a multiclip story so i took the original 1min lowrez mulit-clip video i made here and handed it to both cc & codex with a FAL key... (more below)

experiment with bytedance's new SEED audio model... it's tricky to get consistent audio out of seedance prompts, esp if you want a multiclip story so i took the original 1min lowrez mulit-clip video i made here and handed it to both cc & codex with a FAL key... (more below)

Gavin Purcell

10,572 Aufrufe • vor 25 Tagen

Introducing Stable Audio 2.0 – a new model capable of producing high-quality, full tracks with coherent musical structure up to three minutes long at 44.1 kHz stereo from a single prompt. Explore the model and start creating for free at: Read the blogpost here: (1/3)

Introducing Stable Audio 2.0 – a new model capable of producing high-quality, full tracks with coherent musical structure up to three minutes long at 44.1 kHz stereo from a single prompt. Explore the model and start creating for free at: Read the blogpost here: (1/3)

Stability AI

457,689 Aufrufe • vor 2 Jahren

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

AI at Meta

122,001 Aufrufe • vor 2 Jahren

🚀 Introducing Community Benchmarks on Kaggle! As AI evolves at an unprecedented pace, measuring intelligence requires more than a few AI research labs alone – it requires the imagination and collective expertise of the global community. That’s why we’re launching Community Benchmarks. It lets you build, run, and share custom AI benchmarks that are evaluated on the leading AI models with transparent, reproducible results. Learn more:

🚀 Introducing Community Benchmarks on Kaggle! As AI evolves at an unprecedented pace, measuring intelligence requires more than a few AI research labs alone – it requires the imagination and collective expertise of the global community. That’s why we’re launching Community Benchmarks. It lets you build, run, and share custom AI benchmarks that are evaluated on the leading AI models with transparent, reproducible results. Learn more:

Kaggle

54,516 Aufrufe • vor 6 Monaten

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan

52,119 Aufrufe • vor 9 Monaten