GPT Sol is the first model that has been able to work in 3D Unity/Blender basically autonomously. Since Sunday, I have had an agent building out a noir mystery game from a plan.md. The idea is to have the "voice in your head" be an AI agent trained on... noir mannerism and each NPC will be a unique fine-tune for open ended voice conversations. Sol has created the pre-art, built the levels and rigged the characters. Still really rough but amazing how far we have come in a couple years as I watch it build while I do my work on another monitor. Praying for more resets so it can keep working through the week.show more

Loading Blender MCP takes me forever. I spent ages trying to dig out an eyeball, and now it's basically unusable. Am I doing something wrong, or was this tool overhyped?

I'm not using the MCP, just direct API access. I also decided earlier on to use styled models without eyes (and mouths mostly) because I was unsure of it's ability to do this level of detail work.

@Mihalin1431 So your agent wires in via API to Unity?

@Mihalin1431 Correct. It has asked for computer use a couple times and has trouble with some easy things (like importing from blender with a 90 degree off rotation and spending a long time trying to correct this) but almost all the work is headless through the API.

@Mihalin1431 Damn that's amazing, I'm trying to replicate this right now in @UnrealEngine - but I'll share updates once I get something working.

@Mihalin1431 @UnrealEngine Yeah, I am wondering if Unreal is a better fit for this project as well. I just pointed Sol at Unity because it was already installed on my machine and assumed it would be easier.

Wow this is incredible

Thanks Sana! It's fun waking up and seeing the progress that has been made while you sleep 😂

It’s mind blowing. Hope you are well . ❤️

absolute legend

This is impressive. I'm experimenting with something similar right now but with another theme. How many workhours did it take for Sol to achieve this?

There are 3 threads, working pretty much since Sunday, unless they are blocked (only one agent can use unity or blender at a time). Sunday and most of Monday was mostly pre-dev, storyboards and plot and started locations and characters Monday evening.

That's a great idea! One agent is constantly dishing out 3d models? Do you use external mcps like meshy or do agents do it themselves?

If this is fully autonomous, I expect the games of the next decade to be terrifying.

I tried to get sol to make a simple low poly spaceship thruster in blender with clear source image, it modeled something that looks like a potato getting pegged by a carrot.. So color me unconvinced.

Yeah, many of the first models are horrible (also I am not constrained by super low poly which Sol can have problems with). I have found that having the AI generate art references first, then it can set a goal to keep iterating and improving until it reasonably matches the goal. This bedroom which was still very rough in the video above is one thing it worked on last night

how does it create these models? are the man and woman ful 3d models built in Blender Or are they video generation by video models? Need more details

The intro is a video cinematic. The characters "in game" are build and rigged in blender. It is working on more detailed "shoulder up" models currently to use for the conversational dialogue.

This is impressive, i kind of have the same workflow where i give it access to my project. Sol 5.6 is actually pretty good in Unity. What MCPs are you using? I have Coplay and the Unity official MCP

This is all headless (to the best of my understanding). API access but no MCPs. It does ask for computer use occasionally. Python build script → Blender .blend → validated FBX → Unity import

Yeah that actually works too. I prefer computer use or MCP, that way you can see the changes in realtime. It may be just me but i feel like computer use does consume less tokens than doing API calls

What is with their faces!? Really cool looking so far

@WhatsupFranks It is a style choice form the beginning to overcome tech hurdles. I want all the NPC AI agents to respond in real time but did not think I could convincingly get lipsync and eye movement to generate live so wanted a "faceless cast".

amazing job

Incredible. This process interests me. I created an expert skill in Unity to help me with scripts and computer use, but I didn’t get that far.

this is incredible

hot damn this is impressive 👀

How to connect sol with unity/blender?

It is just api access and not currently MCPs. Python build script → Blender .blend → validated FBX → Unity import

Been looking to get into game dev - I’m curious, what are the specs of your machine do to all of this, especially the ram?

I am currently running on an M4 macbook pro with 128 gigs of ram.

I’ve been using unity mCP to make this game , you mean if I take a step further and hook up blender it can skin it to look like your game ?!

Dude, I built a tool that would help you to plan out large games like this and develop wiki's that contain all the lore and history you need to make coherent game stories. You can use it to develop high fidelity personas(imagine designing a story from the actual perspective of the protagonist) that you can put together in a council to plan and design an entire videogame storyline, game world lore, and design plan, in rich detail you can trust and act on. I really think you'd like it.

Cool, I would be interested. Right now each character has a full system prompt and working on loras for each of them. Testing with text before live audio.

This might help explain how I am building these minds.

impressive. i cannot say i have good results with sol 5.6 for anything visuals-related in gamedev. probably it's user error, but fable seems miles ahead in this regard and is just generally way easier to work with

Interesting! I started with fable (it helped me edit and produce the plan.md) but I found it was too slow and hitting random safety triggers to consistently leave running for 12+ hours at a time. Plus I use fable a lot for work and was on limited tokens for a hobby project.

Dope man, this is all Unity in the video? And blender is just for the models and rigging/animation? Also what plan are you on?

Yep. Also have an API for meshy it can use for harder modeling jobs (which then usually needs cleaned up and simplified in blender. I'm on the $200 Pro plan but no additional cost.

looks terrible lmao

Insanely cool!

Thanks Justin! Always fun to kick the tires on new releases.

Looks great

Thanks!

Hey mate Would you care to share the promotion level you went to to get this result Impressive

Sure! Happy to.

Pre-art? So what you see in this video is not directly from sol?

The cinematics are truly amazing, but the game (just delete it man :D)

Its day 3 of (agent) production :) I'm willing to give them at least a month before I judge it on AAA caliber...

Have you figured out a way to prompt real locations to have it do this?

Not sure the level of detail it could achieve but I am sure this would work. Right now it is building of generated art references so real locations should actually be easier.

interesting, i'm trying to model buildings in unreal engine but it gets away from me.

Is this done on blender?

Blender, Unity and Meshy.

insane

This is so cool - trying to do something in a similar lane but not in blender or unity

That’s actually fantastic

my brain is genuinely hurting rn. please lend a workflow

I’ll play when done

🙌

Does it generate sounds too?

The AI agents have api for sound effects, voice and music but I (poorly) put in the soundtrack for the quick demo.

@PromptCentralX Could you share the soundtrack ?

@PromptCentralX Sure:

We tried using 5.6 sol with blender mcp but the generations were terrible. how did you harness yours together?

I don't use the mcp but just direct api headless access currently. Python build script → Blender .blend → validated FBX → Unity import