I’ve been making a new world building engine in Three.js and thought I’d share some of the crazy milestones I’ve achieved so far… 🧵

First, people need avatars, so I made it so that you can drag and drop VRMs. Pretty simple stuff so far.

Avatars need locomotion, so based on your input a specific “emote” is lazy-loaded, retargeted and animated onto each avatar on the fly.

Skinned mesh animations are expensive due to all the bone matrix math, so we throttle updates based on distance. Far away avatars run animations at lower framerates. This lets us have 150-300 unique, independently animated @forgottenrunes avatars in the world at once.

Next is world building. I added support for dragging and dropping 3D models such as GLB and VOX. You can right click things to move and duplicate them etc.

World building often involves picking a “palette” of models and then placing down hundreds or thousands of them. Doing this the naive way will quickly exhaust our draw calls and CPU limit.

The way Unity handles this is by disconnecting the scene graph from the renderer. I built a new scene graph for ThreeJS that works the same way, by batching and instancing everything together behind the scenes.

We still have this concept of an “object” made up of many meshes, but behind the scenes its updating individual instances in their respective batches.

Now that we can place thousands of objects, it’s a waste of resources to render distant objects at full resolution. I made it so you can “tag” meshes in blender with a LOD distance, letting you render low poly lods far away or completely hide small objects at a certain distance.

Tagging meshes is ultra useful, I added additional tags for physics collision and wind effects too.

It’s now possible to have a ridiculous number of objects in the world, leading to super expensive raycasting. By default, ThreeJS raycasts against everything in the scene.

To fix this, I made the new scene graph insert everything into an octree and ran raycasts against that instead. Regular octrees suffer if you have lots of moving objects, so I had to create my own “loose” octree. Raycasting is now essentially free!

Now that we can make huge amazing looking worlds, we need more interactivity to make things fun. I made it so that each object can have an optional script. You can edit this live in the browser, with other people in realtime.

Each script runs in its own secure sandbox that can only manipulate its own meshes etc. The goal here is to allow people to create, collect and use objects across any world. You can think of each object as being its own web browser tab, completely isolated from other tabs.

I made it possible for scripts to be able to sit idle until needed. This allowed me to place 20k interactive trees that can all be chopped down.

Overall I’m pretty optimistic about where this is heading. If you have any questions I’ll try to answer them all below!