🌟Your static 3D world models are now alive and interactable! 🚀Introducing NeuROK, a neural simulation framework that turns any static 3D object into an interactive 4D asset — no per-category physics, no physical annotations for training. 📄 🧵 1/n

NeuROK takes a static 3D mesh — with no physical or structural annotations — and generates the full 4D trajectory of every vertex under given physical conditions. No material parameters. No category labels. Geometry in, motion out. (2/n)

Traditional simulation workflows often require choosing the right model, crafting the geometry, tuning physical parameters, and iterating until the behavior looks right. NeuROK is designed to be general: one framework covers articulated objects, cloth, elastic bodies, and multi-body systems — no per-category tuning, no separate models. (3/n)

The magic behind this is a centuries-old idea: Lagrangian mechanics. The lesson is: the right choice of coordinates can make a hard dynamics problem much simpler. Traditional simulators often track redundant, over-parameterized coordinates, then pay the cost of enforcing constraints to keep them physically valid. NeuROK instead learns an implicit, compact state space from data — one where those constraints are built in, in a data-driven manner. A simpler state space. Fewer constraints to enforce. (4/n)

Inspired by Lagrangian mechanics, NeuROK learns an object’s configuration space as a latent space from a large-scale 4D dataset. Modeling the dynamics then becomes almost embarrassingly simple: write down the Euler-Lagrange equations on the learned latent manifold. One ODE to model all objects. Check out our interactive illustration: (5/n)

Pass an object mesh into the model, solve the ODE, and you get its 4D dynamics. That’s the whole pipeline. This means you can scan a real room and start interacting with the objects in it. Here’s me interacting with objects in a kitchen (2/F Gates Building at @StanfordAILab @StanfordSVL). (6/n)

… or an embarrassingly messy kitchen like ours :) No special setup. Just a phone scan. (7/n)

… or turning @AlexHe00880585's @Cornell office into a playground (8/n)

… or simply in virtual spaces with artist-created or AI-generated 3D assets! (9/n)

NeuROK will be presented at @CVPR 2026. Come say hi! Many more results on our project page: And check out the paper: Huge thanks to our incredible team: @AlexHe00880585, @yuegao_cs, @zhang_yunzhi, @elliottszwu, and @jiajunwu_cs! (10/10)

Looking forward to the code release!

Amazing work! I think an ultimate demo could be scanning a robot arm and dragging it while keeping the kinematics correct.

This is interesting! Congrats on the release, looking forward to the code

Fascinating direction for world models and simulation.

Exciting work! Quick question for understanding, with a single potential function (eq. 13), what separates elastic (e.g. sponge) and plastic (e.g. laptop) deformations? My intuition says that the laptop should try to return to its initial configuration.

So I can simulate a Newton's cradle instead of doing actual work. Lagrangian mechanics as inductive bias is clever though.

great work, curious whether the learned NOK representation is tied to the underlying mesh representation, or if it could be extendable to photorealistic representations such as 3dgs, assuming lots of training data of course.