Загрузка видео...
Не удалось загрузить видео
Today, we're open sourcing the code behind "The Infinite Crate," our VST/AU plugin that lets you play with Lyria RealTime directly inside your favorite DAW! 🎧🎶 Since we released The Infinite Crate last year, it’s been used by some of our favorite artists - including a wonderful showcase with... show more
20,912 просмотров • 6 месяцев назад •via X (Twitter)
Комментарии: 15

We architected the plugin using JUCE/C++ and React/Typescript - a clever idea presented by @spencersalazar at ADC 2020. This approach ensures latency sensitive operations (audio streaming and processing) happen in a compiled and unmanaged language with a tight clock, while leveraging web technologies for rapid prototyping of the UI (@reactjs, @shadcn, and @tailwindcss). State is synced between Typescript and C++ using Zustand’s state management and nlohmann json. The based frontend also makes it easier for developers to open the codebase in Antigravity (or similar LLM coding tool) and vibecode new interfaces and functionality.

In the near term, we hope to update the plugin to support on-device inference of the Magenta RealTime open-weights model for offline use. In the long term, we hope to support future music models with improved controls, such as audio and MIDI input. We’ll be running a few workshops for people interested in learning to build on the Infinite Crate plugin and the Magenta RealTime model. Join the discussion on our Discord if you’re interested: Looking forward to seeing what new plugins and interfaces you build 🙂

This work has been a labor of love of Ashutosh Desai (@desaiashu). He also releases awesome music through his project Tōsh (IG: @toshbeats), check him out 🎵. Many thanks to Spencer Salazar for his talk on prototyping DAW plugins in web technologies at ADC 2020, JUCE for implementing a C++ to Web/JS bridge in JUCE 8, to the support of artists Sidecar Tommy from Beats Antique (@sidecartommy), Toro y Moi (@ToroyMoi) in exploring the possibilities of DAW integration of our models. Thanks also to the Magenta Team, particularly Noah Constant (@noahconst) for studio experiments and managing development, Ilaria Manco (@Ilaria__Manco), Cassie Tarakajian (@hellothisiscass), Jeff Chang (IG: @instajeffchang), and Adam Roberts (@adarob), Alberto Lalama and Joyce Xie for their work on the API, Nikhil Bhanu for his work on the windows build, and the whole Lyria team that helped contribute to Lyria RealTime.

More info and links in the blog post:

Super cool! I've been playing around with this and it's really fun. Looking forward to exploring the code and maybe making some samples

JUCE + React for audio plugins is such a smart split. Latency-critical paths in C++ (you get it), UI in web tech (faster iteration). One tip: use WebAudio API for the bridge - native audio graph in JS, zero-copy buffers via SharedArrayBuffer. Also, expose JUCE params as React state via a thin IPC layer, not direct DOM manipulation.

now, is a potential use-case me being able to sing along with it, sort of flowing based on the music and my vocals, like I am jamming with a IRL band?

Would you be interested in being interviewed about this project? We have a section on our magazine at that focuses on music-related open-source projects. our platform is also open source :)

@DesaiAshu

🔥

@grok wwhat does this app do?

ok wow, gonna have to try that out in ableton later. what are your favorite artists who use it?

@jesseengel is there some kind of press material for us to work with, or should I just try gather all you have on github?

There's a blog post here:

Thank you Jesse, here's our coverage:


