3D Crowd Generation with Stable-Diffusion and Blender. A thread exploring current simplest approaches, limitations and possible improvements. #stablediffusion #crowdgen #b3d

Any generative-model could work, but stable-diffusion has an awesome ecosystem, and can leverage ControlNet to generate full-body controllable poses. Then is all about good prompting and quality fix (e.g. hd +face-restoration).

Similarly, PIFuHD ( doesn't always generate the best results, but is easy and fast. Potential better alternative are already there, for example ECON ( more powerful but slower.

Biggest limitation is texture. We can project the original stable-diffusion image, but is missing sides and back. Projects exist already for this, for example The idea is to use again generative-models to generate (or complete) a texture given a 3D object

Example: a crowd of knights #stablediffusion #b3d

Example: "sci-fi" crowd #stablediffusion #b3d

I just published an article describing step-by-step the process and tools to achieve the results showcase in the video. Feel free to drop comments, feedback and requests. #stablediffusion #crowdgen #b3d

If you generate them in a Tpose, icon them and then rig/animate them in Mixamo, you will end up with a crowd that moves. And you can program a sequence of looped motions to make them fight, cheer or just chill out

I'm exactly working on that too. Simple process, but can't access Mixamo programmatically, so you gotta rig it manually, which doesn't scale.

detailed tutorial!!! please!!

Wow