Today, we're open sourcing the code behind "The Infinite Crate," our VST/AU plugin that lets you…
@jesseengel9.2K followers
20,963 views
15 Comments
Comments from the original post on X (Twitter)

@jesseengel
We architected the plugin using JUCE/C++ and React/Typescript - a clever idea presented by @spencersalazar at ADC 2020. This approach ensures latency sensitive operations (audio streaming and processing) happen in a compiled and unmanaged language with a tight clock, while leveraging web technologies for rapid prototyping of the UI (@reactjs, @shadcn, and @tailwindcss). State is synced between Typescript and C++ using Zustand’s state management and nlohmann json. The based frontend also makes it easier for developers to open the codebase in Antigravity (or similar LLM coding tool) and vibecode new interfaces and functionality.
22 likes

@jesseengel
In the near term, we hope to update the plugin to support on-device inference of the Magenta RealTime open-weights model for offline use. In the long term, we hope to support future music models with improved controls, such as audio and MIDI input. We’ll be running a few workshops for people interested in learning to build on the Infinite Crate plugin and the Magenta RealTime model. Join the discussion on our Discord if you’re interested: Looking forward to seeing what new plugins and interfaces you build 🙂
12 likes

@jesseengel
This work has been a labor of love of Ashutosh Desai (@desaiashu). He also releases awesome music through his project Tōsh (IG: @toshbeats), check him out 🎵. Many thanks to Spencer Salazar for his talk on prototyping DAW plugins in web technologies at ADC 2020, JUCE for implementing a C++ to Web/JS bridge in JUCE 8, to the support of artists Sidecar Tommy from Beats Antique (@sidecartommy), Toro y Moi (@ToroyMoi) in exploring the possibilities of DAW integration of our models. Thanks also to the Magenta Team, particularly Noah Constant (@noahconst) for studio experiments and managing development, Ilaria Manco (@Ilaria__Manco), Cassie Tarakajian (@hellothisiscass), Jeff Chang (IG: @instajeffchang), and Adam Roberts (@adarob), Alberto Lalama and Joyce Xie for their work on the API, Nikhil Bhanu for his work on the windows build, and the whole Lyria team that helped contribute to Lyria RealTime.
4 likes

@jesseengel
More info and links in the blog post:
5 likes

@JJamesJJulius
Super cool! I've been playing around with this and it's really fun. Looking forward to exploring the code and maybe making some samples
3 likes

@MoodiSadi
JUCE + React for audio plugins is such a smart split. Latency-critical paths in C++ (you get it), UI in web tech (faster iteration). One tip: use WebAudio API for the bridge - native audio graph in JS, zero-copy buffers via SharedArrayBuffer. Also, expose JUCE params as React state via a thin IPC layer, not direct DOM manipulation.
3 likes

@Web3Metal
now, is a potential use-case me being able to sing along with it, sort of flowing based on the music and my vocals, like I am jamming with a IRL band?
1 like

@michelson
Would you be interested in being interviewed about this project? We have a section on our magazine at that focuses on music-related open-source projects. our platform is also open source :)
1 like

@jesseengel
@DesaiAshu

@Moozziii
🔥

@dukeskyrunner
@grok wwhat does this app do?

@YashPnke
ok wow, gonna have to try that out in ableton later. what are your favorite artists who use it?

@audionewsroom
@jesseengel is there some kind of press material for us to work with, or should I just try gather all you have on github?

@jesseengel
There's a blog post here:

@audionewsroom
Thank you Jesse, here's our coverage:
1 like