跳过导航
🥳Excited to share our latest work: "Diff-A-Riff"! 🥁
0:00 / 3:33

🥳Excited to share our latest work: "Diff-A-Riff"! 🥁

Stefan Lattner

@deeplearnmusic1635 位关注者

在 X 上查看
161 个赞下载原帖

17,760 次观看

🥳Excited to share our latest work: "Diff-A-Riff"! 🥁 A Latent Diffusion Model that generates instrumental accompaniments for any musical input, specifically tailored for music producers! It's faster, lighter, and produces superior audio quality. Control via text/audio references. 48kHz sample rate, (pseudo) stereo, ~3Gb memory, takes 6 seconds to generate 90 seconds of music. Trained on a single GPU. 📜 🎶 🎸 "Diff-A-Riff" adapts to any musical input, following the artist's unique style. 🎛️ Optional controls via text prompts, audio references, interpolation slider, pseudo-stereo width and loop intensity. 🎚️ It produces state-of-the-art audio quality indistinguishable from real data by human raters and operates at unprecedented speed. 🧠 "Diff-A-Riff" is smaller and more efficient than previous models thanks to its Consistency Autoencoder, making it accessible and practical for various applications. Big shoutout to my outer space colleagues: Javier Nistal, the Machine in "machine learning" 🚄, Marco Pasini, the neural net whisperer 🤫, Cyran Aouameur, the troubleshootah 🛠️, Maarten Grachten, aka MaartenGPT 🤖. #Teamwork #AI #MusicTech #Innovation Javier Nistal Marco Pasini Cyran Aouameur SonyCSL(Paris)_Music Team

10 条评论

评论来自 X (Twitter) 上的原帖

  • Stefan Lattner 的头像

    @deeplearnmusic

    Update! 🥳 This example demonstrates how to produce a song with Diff-A-Riff starting from a simple template and adding one instrument at a time:

    7 个赞

  • ARGO 的头像

    @AltRealityGameX

    Can't wait to kick this around. So glad to see all the new accomapaniment stuff coming out. This conrner of the tech. Accompaniment, trained musician agents, working towards realtime interactive "jamming" to vibe with automation that doesn't feel like just playing with a looper!

    5 个赞

  • Michael Anslow 的头像

    @MAChirrups

    Very cool stuff!

    3 个赞

  • Devin Arne 的头像

    @devinarne

    Congratulations! This is an amazing tool, the one stem at a time approach is great for musicians/producers as it gives granular control. I could also see this as helpful for film/TV composers to make variations on themes and cues. Hope to be able to use this in the near future!

    3 个赞

  • Janne Spijkervet 的头像

    @JanneSpijkervet

    Now this is exciting!! Well done! Two questions came to mind after a first glance: 1) 50% dropout on audio/CLAP conditioning appeared high to me, was this required? And 2) the impact of the CAE (I’m impressed by the decoder at that number of params!) Nice work!

    3 个赞

  • Florian van Oirschot 的头像

    @florianvo

    Wow! That sounds impressive :)

    3 个赞

  • Matan Gover 的头像

    @matangover

    Awesome work, great results!! One question about inpainting - in the demo audios, the inpainting result sounds slightly different than the original, even in the non-masked region. Is this due to CAE roundtrip?

    1 个赞

  • SGM 的头像

    @234Sagyboy

    @deeplearnmusic Interesting I would love and appreciate to see a similar type of work with tts please look into it Thankyou so much

    1 个赞

  • gregorylent 的头像

    @gregorylent

    @MacTuitui can i mix the output? delete tracks? change levels ??

  • Stefan Lattner 的头像

    @deeplearnmusic

    @MacTuitui Hi Gregory! The model adds one track at a time, giving the user full control over the mix.

    2 个赞