So, you think GPT-4 can't make a complex game... think again! Here's how I used GPT-4, Replit ⠕, MidJourney, Claude, assembling a team of AI assistants, to create a 3D space runner from scratch with ZERO knowledge of Javascript or 3D game programming Follow along for a saga! 🧵

After Snake, seeing @skirano's epic pong recreation, and @javilopen demoing Doom, I wanted to see if I could get GPT-4 to make a fully fledged 3D game. A childhood favorite game of mine was an MS-DOS game called SkyRoads, it was long forgotten. But... did GPT-4 know about it?

Turns out it did! It seemed to be describing the Wikipedia page of the game but that was all... so it was a true test—could we bring this game back to life based on just our understanding of it and a description?

I don't know how to write a line of 3D game programming code, so I asked GPT-4 to pick out how it would make the game, it suggested using BabylonJS, and HTML and CSS. Then I described exactly what we would borrow from SkyRoads and how this version would be different—prompt:

Once I got the code, I pasted it into @Replit the exact way GPT told me too, and it worked! Except... we were FAR from the final product. The green cube, our soon-to-be spaceship, had no ability to turn left or right no way to dodge those pesky obstacles. Well, GPT fixed that🚀

Then, the camera angle was off, we were seeing this cube from the front, but we needed to be behind the spaceship! So I described how the camera needed to look, and GPT fixed that. Turns out it was a simple one line change.

Then came the elusive bug. The spaceship would launch into the abyss if I hit jump+directional keys, crashing the game. GPT-4 and I tried everything for HOURS. I turned to @AnthropicAI's Claude on @poe_platform. In under 60 seconds, Claude solved it.

This also unlocked a new way of working for me—where GPT-4 could be verbose and far more accurate with longer code generations (Claude started adding random lines that we're in the originally pasted text), Claude could help with small fixes. We had a team.

There were plenty of other bugs. The ship would fall through the track, the track would just stop at a certain length, but then the dynamic between me and GPT/Claude became clear: I was the play tester trying to break the game. GPT and Claude do their best to fix it.

Now that the bug was solved, back to the fun part! I needed backgrounds, textures, and a spaceship model for my game. I found my spaceship model on @Sketchfab and asked GPT to give me the code to replace that green cube with my 3D model file.

Then I went to MidJourney for the background and textures, and upscaled them using @pixelmator's super resolution, and its magic brush to make the texture more seamless.

And then asked GPT to make bring all my files in. Then 1 made the game more challenging. Obstacles needed to be of varying width, the track needed to split off to the left and right, I wanted to save the player's best score so they had something to beat.

Finally, went to @epidemicsound to add some music to the game, and then asked GPT-4 to give me the code to add a music player, explosion and game over sounds when you crash. Some classic synth when you're flying your ship, of course!

Other notes: it got to a point where the code was too long for GPT to take in as a question, so I had to use the API playground. A pro tip from @skirano, but GPT actually got better when I told it it was "a world-class BabylonS developer."

And there you have it! In the span of 3 days, with zero 3D game programming knowledge, I was able to recreate an adaptation of one of my childhood favorite games using a combo of Al and existing tools!

Play it here, share your score, and if you have any questions, I'd love to help!

Bigger clip for folks on mobile! :)