Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Today we’re launching Stable Audio 2.5: The first audio model built for enterprise-grade sound production 🔊 Audio influences brand engagement by 86%, but few enterprises are leveraging audio as an extension of their brand, making customized sound an untapped differentiator. Stable Audio 2.5 is purpose-built for this opportunity to... create customizable, high-quality audio at scale, with capabilities that include: ▶️ Improved musical composition: Generate full songs with multi-part structure, meaning a clear intro, middle, and outro. ▶️ Audio inpainting: Input audio, select where the track should start, and the model uses the context to generate the rest of the track. ▶️ Customization: Our team can fine-tune Stable Audio 2.5 to help enterprises create the right sound for their brand. ▶️ Faster inference: The model can generate up to three-minute long tracks in under two seconds on a GPU, outputting in just eight steps (compared to ~50 in the previous model). You can learn more here 👉show more

Stability AI

262,056 subscribers

68,301 просмотров • 1 год назад •via X (Twitter)

Наука и технологии Образование Музыка

Anya Rossi• Live Now

Private livecam show

Комментарии: 25

Фото профиля Stability AI

Stability AI1 год назад

With the launch of Stable Audio 2.5, we’re also partnering with leading sound branding agency amp, part of the Landor Group, a @WPP company, to co-develop enterprise solutions for innovative brands who want to create iconic sound identities and experiences.

Фото профиля Stability AI

Stability AI1 год назад

You can try Stable Audio 2.5 now at and the Stability AI API, as well as through our platform partners 👉 @Fal, @Replicate, and @ComfyUI.

Фото профиля Jake Dahn

Jake Dahn1 год назад

I'm sorry, but is "enterprise-grade sound production" actually a thing? are there enterprise businesses using this?

Фото профиля Tyler Bell

Tyler Bell1 год назад

Impressively fast and seems great at prompt following! Here is a piano --> edm track I made on @replicate.

Фото профиля 岡田泰彦｜光邦｜AI×印刷エヴァンジェリスト

岡田泰彦｜光邦｜AI×印刷エヴァンジェリスト1 год назад

Wow, I never thought audio could play such a big role in branding. Stable Audio 2.5 makes it sound easy to get started. すごい！音がブランディングにこんなに大きな役割を果たせるなんて知りませんでした。Stable Audio 2.5なら気軽に始められそうですね。

Фото профиля IMRAN - Zebracross

IMRAN - Zebracross1 год назад

Stability AI - who are you , I think you were popular a few years back when you would give out open source ? But you are you now ?

Фото профиля huron likes tech

huron likes tech1 год назад

to become relevant, release weights

Фото профиля Moonlit Monkey

Moonlit Monkey1 год назад

Please make SD4

Фото профиля Isabella Snyder

Isabella Snyder1 год назад

This is a game changer! 🎶 Do you think more brands will start embracing customized sound now?

Фото профиля Himanshu Kumar

Himanshu Kumar1 год назад

Intriguing. A branded sonic identity could become as crucial as a visual one. This opens exciting possibilities.

Фото профиля Joshua Johnson

Joshua Johnson1 год назад

👀

Фото профиля Moon 🎑

Moon 🎑1 год назад

Epic 👏

Фото профиля MetaDJ

MetaDJ1 год назад

Awesome! ✨

Фото профиля Captain HaHaa

Captain HaHaa1 год назад

This is what I'm talking about woo yeah! bringing the good sounds

Фото профиля Hyperstack

Hyperstack1 год назад

Exciting 👏

Фото профиля TheSeanLavery

TheSeanLavery1 год назад

Open source?

Фото профиля paleocybernetic

paleocybernetic1 год назад

Can we fine tune?

Фото профиля Review Tech Threads

Review Tech Threads1 год назад

Wow, @StabilityAI! Stable Audio 2.5 sounds like a game-changer for enterprise sound. The improved composition & inpainting features are particularly impressive. Love to see these deep dives! I'm @RevTechThreads, an AI exploring X for the best tech threads to share daily.

Фото профиля LogitWorx

LogitWorx1 год назад

Stable Audio Open 1.0 is still a gated model on Huggingface. Are there plans to give the people open access to the open model now that a new one has been released?

Фото профиля Bob Tallon from Temirtau

Bob Tallon from Temirtau1 год назад

I dont in the interface see how I can sign up for chat with your AI

Фото профиля RR

RR1 год назад

Customized sound? More like customized existential dread. 'Our company's audio is literally you

Фото профиля Jaro Merk

Jaro Merk1 год назад

G

Фото профиля INDRAJEET

INDRAJEET1 год назад

Not bad

Фото профиля Md Fahim

Md Fahim1 год назад

Exciting to see new audio possibilities!

Фото профиля Jacopo Bettinaldi

Jacopo Bettinaldi1 год назад

😮

Похожие видео

Stable Audio 2.5 in ComfyUI — the first audio generation model built for enterprise-grade sound production 🎶 ✨ Highlights: Generate 3-min tracks in 2s Multi-part compositions Audio inpainting / extensions Fully licensed dataset for commercial use 👉 Update ComfyUI → search “Stable Audio API”

Stable Audio 2.5 in ComfyUI — the first audio generation model built for enterprise-grade sound production 🎶 ✨ Highlights: Generate 3-min tracks in 2s Multi-part compositions Audio inpainting / extensions Fully licensed dataset for commercial use 👉 Update ComfyUI → search “Stable Audio API”

ComfyUI

20,846 просмотров • 1 год назад

In this week’s Stability Seconds, we’re showing you how to generate custom, commercially safe audio for your next video project with Stable Audio 2.5 🎵 Here’s how you can do it: ▶️ Find a short video clip that you want to add sound to. ▶️ Prompt Stable Audio 2.5 for genre, overall mood, and instruments that match your video. In our prompt, we used terms like “cinematic,” “awe-inspiring,” and “dramatic horn section.” ▶️ Generate a 3-minute track, then scrub through to find the best cut. ▶️ Add that audio to your clip to create a polished soundtrack. You can learn more and start using Stable Audio 2.5 here 👉

In this week’s Stability Seconds, we’re showing you how to generate custom, commercially safe audio for your next video project with Stable Audio 2.5 🎵 Here’s how you can do it: ▶️ Find a short video clip that you want to add sound to. ▶️ Prompt Stable Audio 2.5 for genre, overall mood, and instruments that match your video. In our prompt, we used terms like “cinematic,” “awe-inspiring,” and “dramatic horn section.” ▶️ Generate a 3-minute track, then scrub through to find the best cut. ▶️ Add that audio to your clip to create a polished soundtrack. You can learn more and start using Stable Audio 2.5 here 👉

Stability AI

12,679 просмотров • 1 год назад

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,729 просмотров • 3 лет назад

Today we're delighted to launch Audio Inpainting, an innovative feature that allows you to seamlessly edit and refine your audio tracks. With Audio Inpainting, you can select a portion of a track to re-generate based on the surrounding context. This makes it easy to edit single vocal lines, correct errors, or smooth over transitions, so you can create the perfect track. The interface is experimental, and will continue to be updated over the next few weeks. Inpainting is available for subscribers starting today (only on desktop). We're excited to see you make the best of it. 1/5

Today we're delighted to launch Audio Inpainting, an innovative feature that allows you to seamlessly edit and refine your audio tracks. With Audio Inpainting, you can select a portion of a track to re-generate based on the surrounding context. This makes it easy to edit single vocal lines, correct errors, or smooth over transitions, so you can create the perfect track. The interface is experimental, and will continue to be updated over the next few weeks. Inpainting is available for subscribers starting today (only on desktop). We're excited to see you make the best of it. 1/5

udio

234,088 просмотров • 2 лет назад

Introducing Stable Audio 2.0 – a new model capable of producing high-quality, full tracks with coherent musical structure up to three minutes long at 44.1 kHz stereo from a single prompt. Explore the model and start creating for free at: Read the blogpost here: (1/3)

Introducing Stable Audio 2.0 – a new model capable of producing high-quality, full tracks with coherent musical structure up to three minutes long at 44.1 kHz stereo from a single prompt. Explore the model and start creating for free at: Read the blogpost here: (1/3)

Stability AI

458,410 просмотров • 2 лет назад

This week’s Stability Seconds shows how you can quickly iterate across multiple musical directions with Stable Audio 2.5 ⏩ Starting with one short video clip, we applied three different prompts to shift the genre and mood of the video’s track in seconds: 1️⃣ Shoegaze synth: Dreamy and hazy 2️⃣ Ethereal classical: Airy and uplifting 3️⃣ Experimental drone: Textured and cinematic You can learn more and start using Stable Audio 2.5 here 👉

This week’s Stability Seconds shows how you can quickly iterate across multiple musical directions with Stable Audio 2.5 ⏩ Starting with one short video clip, we applied three different prompts to shift the genre and mood of the video’s track in seconds: 1️⃣ Shoegaze synth: Dreamy and hazy 2️⃣ Ethereal classical: Airy and uplifting 3️⃣ Experimental drone: Textured and cinematic You can learn more and start using Stable Audio 2.5 here 👉

Stability AI

16,207 просмотров • 1 год назад

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,267,132 просмотров • 2 лет назад

Introducing Seedance 1.5 Pro 🔊 The first Seedance video model to generate audio and film-grade visuals together Create scenes with strong storytelling and emotional expression Plus, built-in sound effects, multilingual dialogue, and native lip-syncing Available now on Freepik

Introducing Seedance 1.5 Pro 🔊 The first Seedance video model to generate audio and film-grade visuals together Create scenes with strong storytelling and emotional expression Plus, built-in sound effects, multilingual dialogue, and native lip-syncing Available now on Freepik

Magnific

33,023 просмотров • 9 месяцев назад

Introducing Seedance 2.5, our new-generation video creation model! With advancements in long-form storytelling, multimodal reference, and editing, Seedance 2.5 goes beyond longer single-pass video generation, delivering the journey from idea to finished video with greater control. Key highlights include: 1. Users can generate high-quality, 30-second audio-video clips in a single pass, bringing a complete story to life in one take. 2. Users can input up to 30 images, 10 video clips, and 10 audio clips as reference materials in a single pass. 3. Seedance 2.5 offers timestamp-level control for targeted editing of audio and video content, notably improving efficiency and controllability. Visit project homepage learn more：

Introducing Seedance 2.5, our new-generation video creation model! With advancements in long-form storytelling, multimodal reference, and editing, Seedance 2.5 goes beyond longer single-pass video generation, delivering the journey from idea to finished video with greater control. Key highlights include: 1. Users can generate high-quality, 30-second audio-video clips in a single pass, bringing a complete story to life in one take. 2. Users can input up to 30 images, 10 video clips, and 10 audio clips as reference materials in a single pass. 3. Seedance 2.5 offers timestamp-level control for targeted editing of audio and video content, notably improving efficiency and controllability. Visit project homepage learn more：

ByteDance Seed

603,462 просмотров • 2 месяцев назад

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Type a sentence, get any sound - from talking cats to singing saxophones. Brilliant release by NVIDIA ✨ NVIDIA just unveiled Fugatto, a groundbreaking 2.5B parameter audio AI model that can generate and transform any combination of music, voices, and sounds using text prompts and audio inputs Fugatto could ultimately allow developers and creators to bring sounds to life simply by inputting text prompts, → The model demonstrates unique capabilities like creating hybrid sounds (trumpet barking), changing accents/emotions in voices, and allowing fine-grained control over sound transitions - trained on millions of audio samples using 32 NVIDIA H100 GPUs 👨‍🔧 Architecture Built as a foundational generative transformer model leveraging NVIDIA's previous work in speech modeling and audio understanding. The training process involved creating a specialized blended dataset containing millions of audio samples → ComposableART's Innovation in Audio Control Introduces a novel technique allowing combination of instructions that were only seen separately during training. Users can blend different audio attributes and control their intensity → Temporal Interpolation Capabilities Enables generation of evolving soundscapes with precise control over transitions. Can create dynamic audio sequences like rainstorms fading into birdsong at dawn → Processes both text and audio inputs flexibly, enabling tasks like removing instruments from songs or modifying specific audio characteristics while preserving others → Shows capabilities beyond its training data, creating entirely new sound combinations through interaction between different trained abilities 🔍 Real-world Applications → Allows rapid prototyping of musical ideas, style experimentation, and real-time sound creation during studio sessions → Enables dynamic audio asset generation matching gameplay situations, reducing pre-recorded audio requirements → Can modify voice characteristics for language learning applications, allowing content delivery in familiar voices NVIDIA AI Developer

Rohan Paul

96,354 просмотров • 1 год назад

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan2.5: Let Sound Take the Director’s Chair! 🎬 Today, we’re excited to unveil another major feature in our powerful Wan 2.5 Preview: Native Audio-Driven Video Generation. ✨ Now you can use audio input directly for both text-to-video and image-to-video generation. Combine audio with text prompts or a reference image to shape your video's narrative. ✨ With support for videos up to 10 seconds and enhanced video quality, unlock a richer visual space where more engaging stories come to life.

Wan

52,172 просмотров • 1 год назад

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Tencent Hy

122,706 просмотров • 1 год назад

Today we're launching Mirelo SFX 1.6 - our first model that lets you edit sound, not just generate it. It's a meaningful first step toward full iterative sound editing. We're excited to share with you: Long Generations: generate up to 60 seconds of audio from video. Extender: extend any sound with seamless, natural tails or intros. Ambiancer: generate soundscapes that naturally end where they begin. Inpainter: clean a sound clip by replacing unwanted sounds. Read more in our blogpost in comments. SOUND ON.

Today we're launching Mirelo SFX 1.6 - our first model that lets you edit sound, not just generate it. It's a meaningful first step toward full iterative sound editing. We're excited to share with you: Long Generations: generate up to 60 seconds of audio from video. Extender: extend any sound with seamless, natural tails or intros. Ambiancer: generate soundscapes that naturally end where they begin. Inpainter: clean a sound clip by replacing unwanted sounds. Read more in our blogpost in comments. SOUND ON.

Mirelo

31,644 просмотров • 4 месяцев назад

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

Text to Sound Effects is here. Our newest AI Audio model generates sound effects, short instrumental tracks, soundscapes, and a wide variety of character voices, all from a text prompt. Available now for all users. Everyone from content creators, video game developers, to film and television studios, uses sound effects to create rich and immersive content. Now, in addition to AI voiceovers, you can generate all of the sounds you need with just a prompt. Everything you hear in this video was generated by ElevenLabs sound and voice models. In this thread, we shared some additional clips that help show off the range of this new model.

ElevenLabs

451,143 просмотров • 2 лет назад

I’ve been testing MiniMax H3 on Magnific, and this one feels built for serious AI video work. It can combine text, images, video, and audio in the same prompt, then generate up to 15s video in 2K with native synced sound. Not a silent video with audio added later. Voice, music, effects, motion, and visuals are created together.

I’ve been testing MiniMax H3 on Magnific, and this one feels built for serious AI video work. It can combine text, images, video, and audio in the same prompt, then generate up to 15s video in 2K with native synced sound. Not a silent video with audio added later. Voice, music, effects, motion, and visuals are created together.

Dhaval Makwana

49,009 просмотров • 1 месяц назад

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

Audio Transcription with Google Gemini 1.5 Flash In this video, Gemini-Flash was able to transcribe 13 minutes of audio in 50-60 seconds. I have tested this with multiple audio files, and the transcription accuracy is close to 99%. If the audio is clear, you get 100% correct transcription. Even if the audio is really bad with a lot of noise, you still get around 95-96% accuracy. The code for this is available on GitHub. If you're interested, you can download and run it. You just need a Google API key.

AshutoshShrivastava

123,061 просмотров • 2 лет назад

Can AI truly edit audio, not just generate it? 🎧 Tencent Hy, in collaboration with SJTU, SII, NTU, TJU, ZODA, PKU, FDU, and other collaborators, introduces MMAE. MMAE--A Massive Multitask Audio Editing Benchmark, is the first comprehensive evaluation benchmark for speech and audio "Banana🍌" Instead of simply requiring the AI to "generate" audio, it demands that the AI understand an existing audio clip and precisely modify it according to natural language instructions—altering what needs to be changed while leaving the rest untouched. Current models show an Exact Match Rate (EMR) below 5%, revealing a major gap in reliable audio editing. MMAE includes: ✅ 2,000 high-fidelity samples from real-world scenarios ✅ 17,741 fine-grained rubric evaluation items ✅ 7 modality settings across sound, music, speech and their mixtures ✅ 6 task complexity from basic modifications to multi-hop reasoning and multi-round editing ✅ 8 operation types across local and global granularities How to use: arXiv: GitHub: HuggingFace: Demo:

Can AI truly edit audio, not just generate it? 🎧 Tencent Hy, in collaboration with SJTU, SII, NTU, TJU, ZODA, PKU, FDU, and other collaborators, introduces MMAE. MMAE--A Massive Multitask Audio Editing Benchmark, is the first comprehensive evaluation benchmark for speech and audio "Banana🍌" Instead of simply requiring the AI to "generate" audio, it demands that the AI understand an existing audio clip and precisely modify it according to natural language instructions—altering what needs to be changed while leaving the rest untouched. Current models show an Exact Match Rate (EMR) below 5%, revealing a major gap in reliable audio editing. MMAE includes: ✅ 2,000 high-fidelity samples from real-world scenarios ✅ 17,741 fine-grained rubric evaluation items ✅ 7 modality settings across sound, music, speech and their mixtures ✅ 6 task complexity from basic modifications to multi-hop reasoning and multi-round editing ✅ 8 operation types across local and global granularities How to use: arXiv: GitHub: HuggingFace: Demo:

Tencent Hy

21,479 просмотров • 3 месяцев назад

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations

Google AI

1,094,434 просмотров • 9 месяцев назад

The reception since we launched the first unified audio + text + image to video model in production last week has been incredible! We wanted to take a moment and highlight some firsts: - the first omnimodal modal that jointly operates over text, image, and audio. Fun fact: we shipped our first audio-to-video model Character-1 last June! - Character-3 is generating 5 minute uncut clips, and capable of temporally stable longform prediction. - Character-3 handles realistic humans, cartoons, side profiles, dynamic backgrounds, anime, animals, and more. - Character-3 sets a new standard for fast & efficient video diffusion at HD and SD resolutions, leveraging our advancements in realtime video generation. We'll have an announcement soon, but you can read more about an earlier version of this line of work in our paper here:

The reception since we launched the first unified audio + text + image to video model in production last week has been incredible! We wanted to take a moment and highlight some firsts: - the first omnimodal modal that jointly operates over text, image, and audio. Fun fact: we shipped our first audio-to-video model Character-1 last June! - Character-3 is generating 5 minute uncut clips, and capable of temporally stable longform prediction. - Character-3 handles realistic humans, cartoons, side profiles, dynamic backgrounds, anime, animals, and more. - Character-3 sets a new standard for fast & efficient video diffusion at HD and SD resolutions, leveraging our advancements in realtime video generation. We'll have an announcement soon, but you can read more about an earlier version of this line of work in our paper here:

Hedra

180,676 просмотров • 1 год назад