Environment generation is the missing scaling axis for embodied AI. Introducing SimWorld Studio: a self-evolving factory for endless interactive 3D env where agents act, fail & learn. Env-agent co-evolvution improves navigation success 50% → 90%. From a prompt, our SimCoder writes code to automatically build an interactive world. Agents... train inside it. And their performance shapes the next world.show more

So excited to see what future research directions are explored in SimWorld! #EmbodiedAI #MachineLearning #UCSD #Simulation

Digital agents scale because they have scalable sandboxes: code, web, computer-use. Embodied agents need the same — but in 3D worlds. To train capable embodied agents, environments must be: diverse interactive physically grounded verifiable standardized learning interfaces And crucially: able to co-evolve with the agents.

Today, most embodied environments are still manually built or procedurally templated. This makes them hard to scale. Meanwhile, many 3D generation systems can create beautiful scenes — but not executable environments with tasks, rewards, resets, observations, and agent interaction. That is the gap SimWorld Studio aims to close.

At the core of SimWorld Studio is SimCoder. SimCoder is a tool- and skill-augmented coding agent that writes engine-level code in Unreal Engine 5. Given a language or image prompt, it builds a physically grounded 3D environment that agents can interact with. Prompt → code → UE5 world.

But generation alone is not enough. The world must actually work. So SimCoder uses verifier feedback — compilation errors, physics checks, collision checks, VLM critiques, and task validation — to debug and refine the generated environment. The world is not just generated. It is tested, verified, and improved.

SimCoder also self-evolves. When it fixes a mistake, the fix can become a reusable skill. When it builds a useful component, the component can become a reusable tool. Over time, SimCoder accumulates a growing library of environment-building skills.

The output is not a static 3D asset. Each generated world can be exported as a Gym-style environment with: reset observation action reward termination So embodied agents can directly enter the world, interact with it, and learn from experience.

This makes co-evolution possible: An embodied agent learns inside the world generated by SimCoder. Its success rate, failures, and behavior traces become feedback to SimCoder. The next world is generated based on what the agent can or cannot do. If the agent succeeds too easily, SimCoder can make the next world harder. If the agent fails completely, SimCoder can generate an intermediate challenge. The curriculum is no longer fixed. The world adapts to the agent’s capability frontier.

This is the central vision of SimWorld Studio: not just generating 3D scenes, but building a self-evolving environment factory for embodied AI. Describe a world. Generate it. Verify it. Train an agent inside it. Use the agent’s behavior to shape the next world. Website:

Github: Website: Arxiv paper: Huggingface paper: Youtube link:

do think this is another promising direction

Doesn’t NVidia do this?

Will this work for building games? Cuz I'm tired of dragging assets and agents that don't understand creativity...

Yes. And We're adding more features to support building more complex games autonomously.

Awesome looking forward to trying it

cool

Look this video . it's real and better then thi