Many physical systems are high-dimensional, but we only really care about some low-dimensional subspace. Our latest work shows how to fit these subspaces as small neural maps automatically, *without* any data as input, just the energy function. Read on to learn how! (1/N) 🧵

Reduced order modeling seeks to identify & parameterize significant low-dimensional subspaces for high-dimensional systems. Neural nets are a natural representation, but data-driven approaches are tricky: for most systems, data doesn't exist and is hard to collect! (2/N)

To be clear, we're talking about 'overfit' neural networks here: a tiny network fit individually to each system, not pretrained on a giant dataset or anything like that. Think of it like 'baking' a network as an acceleration structure for your system. (3/N)

Ok, so how do we fit these nets? We can start by minimizing physical potential energy on randomly sampled latent subspace states. However, the network just collapses to the lowest-energy configuration! Remember, there's no data term here that it has to stretch to fit. (4/N)

Our simple solution is to borrow another classic ML tool, and introduce an *isometric regularizer*, which says that the distance between latent vectors should be roughly preserved when pushed through the map. This is the secret sauce that makes it work! (5/N)

Now we can automatically fit small networks to any system you can simulate. Our architectures, are just simple fast MLPs. (6/N)

We can even condition the networks on additional parameters, such as boundary conditions and stiffness, and fit shared subspaces that adapt when you vary the parameters. (7/N)

This formulation works out-of-the-box for systems far from the usual deformable bodies. Here, we model a rigid body kinematic mechanism with pin penalties at joints, parameterized by the entries of a transformation matrix. Subspace training finds the smooth motion! (8/N)

Importantly, our low-dim subspace maps are a true 'dense' parameterization: every point in the subspace corresponds to a significant configuration. This makes many downstream tasks easy: for instance you can keyframe animate complex systems via a spline in the subspace! (9/N)

We even show a preliminary application as a data generator: if you have a _supervised_ physics learning setup, but no data, you can fit our subspace and use it to sample training data. (10/N)

This #SIGGRAPH2023 paper is "Data-Free Learning of Reduced-Order Kinematics", with Cristian Romero, @_AlecJacobson, Etienne Vouga, @paulkry, @diwlevin, and @JustinMSolomon. Come see it Wed @ 2pm in the 'Mos Def' session (11/N)

Huge shout out to the Bellairs Institute Workshop on Computer Animation, where this project began as a workshop collaboration! (12/12) - project: - paper: - code: