Introducing Gizmo, a simulation-authoring agent that turns text and reference images into structured, editable 3D scenes for robotics workflows. We're opening the public beta today.

Gizmo accepts text prompts, one or more reference images, or a combination of both. It works best when the asset or scene is clearly defined.

Gizmo represents assets using geometric primitives, rather than treating them as opaque meshes. This produces structured geometry that is easier to edit, resize, articulate, and configure with material and physics properties.

We’ve been fortunate to collaborate with some of the best researchers in simulation and robotics while building Gizmo. Our goal is to push the frontiers of 3D simulations and sim-to-real, and make high-quality environments far more accessible for training and evaluation.

Over the next few weeks, we plan to release: - scene reconstruction from monocular video, with articulated assets - additional tools designed to make existing simulation workflows easier

Gizmo is the world authoring layer for simulations. We currently support exports to @nvidiaomniverse Isaac, @GoogleDeepMind MuJoCo and @OpenRoboticsOrg Gazebo. Scenes and assets include: - real-world scale - semantic labels - collision geometry and affordances - joints and physics parameters

Try the public beta here: Gizmo is available as a web app and an API that you can integrate into your agentic workflows. If you’re working on simulation, sim-to-real, or robot learning, we’d love to explore how Gizmo could support your research.

show a comparison of a model fine tuned for a new task in the sim and the translation of that task into the real world compared to issac sim. would this be more sample efficient during the translation process, more generalizable, or easier to setup in comparison?

We're releasing a report on this in the next couple of weeks! But to answer your question, the immediate advantage today is faster setup and iteration. We're not building the physics solvers, Gizmo is the middle layer for quickly building and iterating on env used in isaac or MuJoCo. The broader goal is to reduce overfitting to a small set of manually authored scenes by generating more diverse variations around real-world failure cases

very cool man. hyped for the report and super cool product. congrats on the launch

please give me more credits, i loved using gizmo

this is awesome. any plans on a place for ppl to share sim spaces?

In the works!

I know it's silly but, any plan of open sourcing ?

Open sourcing ~100 validated diverse scenes for Isaac sim, mujoco, and gazebo!

Great, hope to use it in the near future

structured 3D output format? that's the part i wanna see

the physics, articulation, collision checklist is the part robotics people will actually stop for. pretty 3d scenes are easy, scenes a robot can train against are the hard part, and structured plus editable is what separates this from every text-to-3d demo on the timeline. sent a dm!

🙄🤨🤔