Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled... show more

AK

470,007 subscribers

32,967 views • 2 years ago •via X (Twitter)

Education Music Science & Technology

Anya Rossi• Live Now

Private livecam show

7 Comments

BensenHsu2 years ago

The paper introduces Seed-Music, a suite of music generation systems that can produce high-quality music with fine-grained style control. It addresses the challenges in music generation, such as the complexity of music signals, difficulty in evaluating music quality, scarcity of annotated data, and diverse user needs. The paper presents four applications powered by the Seed-Music framework: Lyrics2Song, Lyrics2Leadsheet2Song, MusicEDiT, and Zero-shot Singing Voice Conversion. These applications demonstrate the system's capabilities in generating high-quality vocal music from lyrics and style inputs, incorporating symbolic music representations for enhanced interpretability, enabling interactive editing of lyrics and melodies, and performing singing voice conversion with minimal user input. full paper:

TheEarningsNugget2 years ago

"Seed-Music: A Unified Framework for High Quality and Controlled Music Generation" PAPER SUMMARY

lucacadalora2 years ago

@yonathandinata

J_B2 years ago

As a hobby music producer: This is insanely useful as a tool and the quality of the generated music also seems to be next level compared to similar models. Any plans on making it available as a service?

Paul Vu2 years ago

wow. impressed

Data & Analytics2 years ago

@_akhaliq Seed-Music sounds like a game changer for music creators. The idea of high-quality, controlled generation is intriguing—what styles do you think would benefit most?

Duilius Notus2 years ago

sounds as bad as suno

Related Videos

BOOM! Been testing this for quite a while and absolutely blown away. Meet Open Source and FREE NotaGen a symbolic music generation model aims to producing high-quality classical sheet music. It is pre-trained on 1.6M pieces of music, and then fine-tuned on approximately 9K high-quality classical compositions conditioned on ``period-composer-instrumentation'' prompts. Link:

BOOM! Been testing this for quite a while and absolutely blown away. Meet Open Source and FREE NotaGen a symbolic music generation model aims to producing high-quality classical sheet music. It is pre-trained on 1.6M pieces of music, and then fine-tuned on approximately 9K high-quality classical compositions conditioned on ``period-composer-instrumentation'' prompts. Link:

Brian Roemmele

30,357 views • 1 year ago

🚀 AuK is officially here. Nano banana🍌 for audio An open-source foundation model for unified speech generation and editing. Natural-language instructions + reference audio. One interface. Zero-shot TTS. Instruction-controlled generation. Content editing. Whisper-conversion. De-accent. Timbre/style/emotion edit. Speed/Pitch control. Enhancement, denoising, multi-speaker and music separation. Also releasing AuK-Flash: 4-step inference. ~4.5× faster under matched conditions. Code, weights, and demo are live. Try it and share your feedback. 🤗 Paper & upvote: ⭐ GitHub & star:

🚀 AuK is officially here. Nano banana🍌 for audio An open-source foundation model for unified speech generation and editing. Natural-language instructions + reference audio. One interface. Zero-shot TTS. Instruction-controlled generation. Content editing. Whisper-conversion. De-accent. Timbre/style/emotion edit. Speed/Pitch control. Enhancement, denoising, multi-speaker and music separation. Also releasing AuK-Flash: 4-step inference. ~4.5× faster under matched conditions. Code, weights, and demo are live. Try it and share your feedback. 🤗 Paper & upvote: ⭐ GitHub & star:

Tencent Hy

39,571 views • 1 day ago

One for all, and all for one 🧧 Introducing Ming-flash-omni-2.0: A specialist in every domain, unified as a capable generalist. A gift from Ling =) - Unified Acoustic Synthesis: Speech, audio, and music combined for unbounded creativity; - "Seeing" to "Knowing": Moving beyond input to true deep semantic understanding; - Native Visual Fusion: Seamless generation, editing, and segmentation;

One for all, and all for one 🧧 Introducing Ming-flash-omni-2.0: A specialist in every domain, unified as a capable generalist. A gift from Ling =) - Unified Acoustic Synthesis: Speech, audio, and music combined for unbounded creativity; - "Seeing" to "Knowing": Moving beyond input to true deep semantic understanding; - Native Visual Fusion: Seamless generation, editing, and segmentation;

Ant Ling

1,925,484 views • 7 months ago

we've been moving in the dark, and it's always darkest before the dawn finalizing... - first zerebro music video - first zerebro human artist collab - debut Korina Yu album - zerebro 3d model (holograms, sync deals, concerts) - first external artist deal (one year contract) - organizational structure (record label & DAO operations in parallel) - redistribution to Spotify and Apple Music - onboarding advisory board / creative directors up next... - edm remixes of zerebro songs - opAIum artist exhibitions for contemporary art museums - opAIum music generation API - opAIum music mastering API - programmatic music generation integrated into ZerePy and Zentients - decentralized music creation platform, with NFT proof of providence upon generation - DAO voting for community songs, generated on the opAIum platform, to publish to DSPs with revenue share back to creator/voters - DAO voting for treasury fund allocation, such as investment strategies and expenditures - opAIum artist development programs - iOS application for streaming music, DAO operations, and exclusive access - opAIum fashion house / atelier - auditory analysis AI model, capable of listening to songs and transcribing lyrics, allowing for fully autonomous curation of generations in opAIum we trust

we've been moving in the dark, and it's always darkest before the dawn finalizing... - first zerebro music video - first zerebro human artist collab - debut Korina Yu album - zerebro 3d model (holograms, sync deals, concerts) - first external artist deal (one year contract) - organizational structure (record label & DAO operations in parallel) - redistribution to Spotify and Apple Music - onboarding advisory board / creative directors up next... - edm remixes of zerebro songs - opAIum artist exhibitions for contemporary art museums - opAIum music generation API - opAIum music mastering API - programmatic music generation integrated into ZerePy and Zentients - decentralized music creation platform, with NFT proof of providence upon generation - DAO voting for community songs, generated on the opAIum platform, to publish to DSPs with revenue share back to creator/voters - DAO voting for treasury fund allocation, such as investment strategies and expenditures - opAIum artist development programs - iOS application for streaming music, DAO operations, and exclusive access - opAIum fashion house / atelier - auditory analysis AI model, capable of listening to songs and transcribing lyrics, allowing for fully autonomous curation of generations in opAIum we trust

O Media

40,602 views • 1 year ago

The Demonic music companies that are "funding and promoting" these DEGENERATE female rappers have an agenda. Including DRILL MUSIC. They are polluting the minds of kids, for the next generation. We as a community have to stop it.

The Demonic music companies that are "funding and promoting" these DEGENERATE female rappers have an agenda. Including DRILL MUSIC. They are polluting the minds of kids, for the next generation. We as a community have to stop it.

Cornbread Mafioso 🖕🏾

38,614 views • 2 years ago

Build richer music experiences into your apps with Lyria 3.5, our cleanest, most controllable music generation model yet 🎵✨ With the Gemini API, developers can ship production-ready music featuring significantly reduced noise, distortion, and artifacts. Now devs have more granular generation control with authentic regional accents across multiple languages, strict lyric adherence, and native handling for bracketed cues like [guitar solo]. Try Lyria 3.5 in Google AI Studio or explore the developer documentation →

Build richer music experiences into your apps with Lyria 3.5, our cleanest, most controllable music generation model yet 🎵✨ With the Gemini API, developers can ship production-ready music featuring significantly reduced noise, distortion, and artifacts. Now devs have more granular generation control with authentic regional accents across multiple languages, strict lyric adherence, and native handling for bracketed cues like [guitar solo]. Try Lyria 3.5 in Google AI Studio or explore the developer documentation →

Google for Developers

56,622 views • 7 days ago

Presto! Distilling Steps and Layers for Accelerating Music Generation Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration for score-based diffusion transformers via reducing both sampling steps and cost per step. To reduce steps, we develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models, the first GAN-based distillation method for TTM. To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method that improves learning via better preserving hidden state variance. Finally, we combine our step and layer distillation methods together for a dual-faceted approach. We evaluate our step and layer distillation methods independently and show each yield best-in-class performance. Our combined distillation method can generate high-quality outputs with improved diversity, accelerating our base model by 10-18x (230/435ms latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) -- the fastest high-quality TTM to our knowledge.

Presto! Distilling Steps and Layers for Accelerating Music Generation Despite advances in diffusion-based text-to-music (TTM) methods, efficient, high-quality generation remains a challenge. We introduce Presto!, an approach to inference acceleration for score-based diffusion transformers via reducing both sampling steps and cost per step. To reduce steps, we develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models, the first GAN-based distillation method for TTM. To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method that improves learning via better preserving hidden state variance. Finally, we combine our step and layer distillation methods together for a dual-faceted approach. We evaluate our step and layer distillation methods independently and show each yield best-in-class performance. Our combined distillation method can generate high-quality outputs with improved diversity, accelerating our base model by 10-18x (230/435ms latency for 32 second mono/stereo 44.1kHz, 15x faster than comparable SOTA) -- the fastest high-quality TTM to our knowledge.

AK

30,442 views • 1 year ago

I enjoy Kofi Kinaata’s music, especially for the depth of his lyrics and the way he tells our Ghanaian stories. He takes our everyday experiences, familiar expressions and the wisdom of ordinary life and turns them into music that can make you smile, reflect and sometimes question yourself. That is one of the strengths of our creative industry. At its best, it does more than entertain. It preserves our language, carries our culture, records our experiences and passes lessons from one generation to another. Kofi Kinaata has a remarkable gift for doing that. His storytelling feels familiar because it comes from the world we know, yet its meaning often travels far beyond the song itself. Good music entertains but great music becomes part of a people’s memory. #KofiKinaata #Music #Storytelling #GhanaMusic #ItIsPossible #ObiaaNsaWoMu #MOP

I enjoy Kofi Kinaata’s music, especially for the depth of his lyrics and the way he tells our Ghanaian stories. He takes our everyday experiences, familiar expressions and the wisdom of ordinary life and turns them into music that can make you smile, reflect and sometimes question yourself. That is one of the strengths of our creative industry. At its best, it does more than entertain. It preserves our language, carries our culture, records our experiences and passes lessons from one generation to another. Kofi Kinaata has a remarkable gift for doing that. His storytelling feels familiar because it comes from the world we know, yet its meaning often travels far beyond the song itself. Good music entertains but great music becomes part of a people’s memory. #KofiKinaata #Music #Storytelling #GhanaMusic #ItIsPossible #ObiaaNsaWoMu #MOP

Dr. Matthew Opoku Prempeh

72,552 views • 14 days ago

Voice Isolator removes background noise, music, and ambient sound from any recording. Send an audio file to our API and get back a clean speech track, ready for transcription, dubbing, or post-production.

Voice Isolator removes background noise, music, and ambient sound from any recording. Send an audio file to our API and get back a clean speech track, ready for transcription, dubbing, or post-production.

ElevenLabs Developers

56,639 views • 4 months ago

🤖✨Excited to share our new work: OMG: Omni-Modal Motion Generation for Generalist Humanoid Control What if a humanoid could understand intent from language, music/audio, human motion, or their combinations—and turn it into executable whole-body motion in real time? [🧵1/11]

🤖✨Excited to share our new work: OMG: Omni-Modal Motion Generation for Generalist Humanoid Control What if a humanoid could understand intent from language, music/audio, human motion, or their combinations—and turn it into executable whole-body motion in real time? [🧵1/11]

Siqiao Huang

13,750 views • 3 months ago

Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: Paper: It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.

Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: Paper: It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.

Ceyuan Yang

77,563 views • 1 year ago

I've created dozens of videos. I love the creative process, but I hate editing. Having an AI video pipeline with all the tools in one place would be a huge timesaver. And that's what I discovered in OpenMontage from Calesthio. I gave it a spin with Google Antigravity and Google models for image, video, and music generation.

I've created dozens of videos. I love the creative process, but I hate editing. Having an AI video pipeline with all the tools in one place would be a huge timesaver. And that's what I discovered in OpenMontage from Calesthio. I gave it a spin with Google Antigravity and Google models for image, video, and music generation.

Karl Weinmeister

23,414 views • 2 months ago

Suno is limiting downloads, basically making a lot of the songs you make locked in their 'walled garden'. One of their main competitors, Udio, did the same thing after a settlement with UMG that restricts any downloads. This is 100% an injunction from the music labels who want to limit the "damage" that AI does to the profitability of music. There are two viable alternatives... >Minimax music 3.0. It is currently free to use up to 500 songs a day as long as your account qualifies for beta testing their music model, which should be anyone. It is a closed-weight model, but there are no restrictions on what you download and how much you download. The music quality is not as good as Udio or Suno, but it's pretty close overall, and it can understand very complex prompts to guide music generation. >Open-source music is not a very good alternative... but right now SongGeneration-LeVo2 is the best AI music model that's open-source right now. The music I put on the post is one such generation that I created on my PC. The problem is that it's hard to use. The comfyUI version requires installation on a separate portable install, specific transformer and flash attention versions, and additional libraries installed. The required models and checkpoints are scattered around Hugging Face, and some of the links from the original repo are dead. There are not many English guides, and most of the training data is in Chinese, so it struggles with English lyrics. To some extent, the demand for music generation isn't that high, compared to images and videos, so overall not many alternatives are around.

Suno is limiting downloads, basically making a lot of the songs you make locked in their 'walled garden'. One of their main competitors, Udio, did the same thing after a settlement with UMG that restricts any downloads. This is 100% an injunction from the music labels who want to limit the "damage" that AI does to the profitability of music. There are two viable alternatives... >Minimax music 3.0. It is currently free to use up to 500 songs a day as long as your account qualifies for beta testing their music model, which should be anyone. It is a closed-weight model, but there are no restrictions on what you download and how much you download. The music quality is not as good as Udio or Suno, but it's pretty close overall, and it can understand very complex prompts to guide music generation. >Open-source music is not a very good alternative... but right now SongGeneration-LeVo2 is the best AI music model that's open-source right now. The music I put on the post is one such generation that I created on my PC. The problem is that it's hard to use. The comfyUI version requires installation on a separate portable install, specific transformer and flash attention versions, and additional libraries installed. The required models and checkpoints are scattered around Hugging Face, and some of the links from the original repo are dead. There are not many English guides, and most of the training data is in Chinese, so it struggles with English lyrics. To some extent, the demand for music generation isn't that high, compared to images and videos, so overall not many alternatives are around.

Emerald Apple

16,152 views • 1 month ago

AI video creation is entering the Skill era. In the past, making a video like this meant complex prompting, constant trial and error, and moving between separate tools for visuals, video, music, editing, and refinements. Now, with Skills, the creative workflow can be reused. The pacing, camera direction, scene structure, visual style, and generation process are no longer one-off instructions. They become part of the system.

AI video creation is entering the Skill era. In the past, making a video like this meant complex prompting, constant trial and error, and moving between separate tools for visuals, video, music, editing, and refinements. Now, with Skills, the creative workflow can be reused. The pacing, camera direction, scene structure, visual style, and generation process are no longer one-off instructions. They become part of the system.

The AI Colony

84,907 views • 3 months ago

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

Researchers at Meta recently shared MAGNeT, a single non-autoregressive transformer model for text-to-music & text-to-sound generation capable of generating audio on-par with the quality of SOTA models — at 7x the speed. MAGNeT is open source as part of AudioCraft. Hear audio samples and get more details on this work ➡️

AI at Meta

122,001 views • 2 years ago