Startracker 🔺
@startracker • 5,768 subscribers
Vibe coding, creative technology, applied AI. Building agentic pipelines for creative, video and online game production.
Shorts
Videos
I MADE THE BEST 2D SPRITE AI PIPELINE EVER! After testing so much, doing tons of math, trying various different methods I finally got truly the cleanest sprite result I could ask for! You should bookmark this post! ✅ I think this is the happiest 'aha' moment I've had since starting this journey. Day 48 of making a custom webGL game engine using Cursor 🔽🔽🔽 I've been trying to solve sprites since I started this project. I came close a couple of times but it was always something - either dynamic lights wouldn't work, or the genAI results would make them grainy, or I would have to have incorrect alpha etc. After so much struggling, wanting to give up, thinking it's impossible - it was finally done! Perfect matte interpolation of black & white backgrounds resulting in a clean outline of the sprite, half transparent effects keeping their full quality. But this was not enough for me. I wanted more. I wanted those dynamic lights for the most badass anime effects that I'll be creating for the next many weeks! And somehow, I managed to come up with the perfect balance that is nice to the eye with almost no visible imperfections or artifacts when using any kind of dynamic lights. See for yourself! 🔽🔽🔽 How does this magic happen? 1⃣ First, the animation is created using Kling AI. I found it the best to keep the quality anime 2D style I wanted, but I guess others may work as well. 2⃣ Then, the frames are carefully extracted at lossless quality and processed via automation through Nano Banana pass to create an opposite background (white->black or black->white). 3⃣ Once we have a proper spritesheet, I then used MiDaS to compute the depth maps, and then algorithmically create PBR maps for lighting. The remaining depth maps can also be used for various effects, although haven't explored it much yet. 4⃣ Finally, all elements are cut into single frames and overlapped with their respective normal maps. If done right, they're pretty much ready for runtime. I like my heroes to be 512 to truly stand out like fighting game characters, but all of this works with far smaller or bigger sprites. Mind you, it's important to note that Nano Banana has a max output of 4k. Therefore you don't want to make your atlases larger than 4k - which is still quite a lot since it allows you 8x8 512 frames. 💡💡💡 I believe the output quality is beyond anything currently available, and I'll be looking into packaging the whole thing into a web app so people can use it. If you find this useful, please repost it! I really wanted to share this method to the world. Thank you for reading! Look forward to day 49 update!
Startracker 🔺145,925 views • 3 months ago
GUYS I SOLVED IT! It simply works! Sprites are solved! I figured if I separate the video into 4 equal quadrants based on the total size as an 1:1 video I could anchor all maps to a central pixel of the diffuse map! Cursor+Codex never fail - as long as you tell them nicely EXACTLY how to solve it! I'm sooo hyped to refine this and integrate it properly into the engine! Look forward to tomorrow! We might have our first unit appear!
Startracker 🔺115,878 views • 4 months ago
FORGET MY PREVIOUS POST, LET ME BLOW YOUR MINDS! 🤯NEW METHOD! AI 2D SPRITES ARE SOLVED. PERIOD. Day 50 of building a webGL game engine using Cursor 🔽🔽🔽 I was wrong. The previous system had flaws. It wasn't the best. If you use it on VFX heavy sprites, it would start falling apart - either via grain/artifacts, or displacement. But I just couldn't give up. I knew I needed something more. This was the limit the models could give me. So I consulted science: Zheng et al., "Bilateral Reference for High-Resolution Dichotomous Image Segmentation" (CAAI AIR 2024, arXiv:2401.03407) - BiRefNet If any of the brilliant minds who wrote this paper read this post, I'm extremely thankful for your research. It made me solve a really big problem with 2D sprites, and I'm extremely grateful! Thank you! Please reach out if you see this! Now, how does this solution work: 1⃣ Generate your Kling animation (still king for anime 2D style). 2⃣ Run BiRefNet (HR-matting variant rocks) → get solid character alpha. 3⃣ Compute simple brightness/luminance alpha (luma → alpha curve, easy in any tool). 4⃣ Final alpha = max(BiRefNet_alpha, brightness_alpha) — that's it. No fancy weights needed. 5⃣ Feed to your engine / Comfy / whatever. Dynamic lights now play nice with zero artifacts. Mind you, this solution requires an nVidia GPU to run local inference. Alternatively, you can get it done on Colab likely with the free tier! You can try CPU, but no guarantees. Try a lighter model like BiRefNet_lite-2K. Demo is purposely in 30 fps even though I wouldn't run an anime 2D game above 12 fps - for both style and performance. Just wanted to show you the absolute results. Please bookmark this! ☑️ If you love 2D and sprite work, this is truly the best way, the best method I could find. Use it, steal it. Enjoy it! Repost if it saves your workflows too! I think I need a break... almost 40 hours straight solving this. Next are dynamic shadows, and I'm not looking forward to that...
Startracker 🔺71,082 views • 3 months ago
Day 33 of vibe coding a game+engine using Cursor So I approached character animation from a different perspective this time. Instead of 3D -> 2D I went for native 2D via image to video method. This gives us much richer animations that the limited ones we get with 3D! Eliza is really cute with these! The downside is the time it takes to get them all done, even with Cursor skills/automation. Which ones did you like more? Check the previous posts for the 3D ones!
Startracker 🔺33,327 views • 4 months ago
I vibe coded and built a sprite animation pipeline 🛠️ (Day 22 of making the engine+game) ⬇️ Watch the video if you don't wanna read the wall of text - it directly shows what I do. Shoutout to Jidé ✨ for showing me a paper on black/white combination to get alpha, it's the cleanest method yet, and to Cursor for enabling this entire journey. If you prefer the wall of text here you go: The hardest part of using general image models for 2D sprites isn’t getting a nice-looking frame, it’s getting consistent motion across a whole sprite sheet. You can fake a sheet, but frames won’t align, timing drifts, and you end up with weird artifacts. Even if you manually cut frames + interpolate, the animation often looks “off” because each frame is basically a new interpretation, not the same character evolving over time. This is especially noticeable with public API models like gpt-image-1.5 and Nano Banana. Some custom LoRAs for open models exist, but this is intended for less techy folks. My workaround: use a video model first, then post-process into a sprite sheet. Render the animation over a solid background (white/black/magenta/green), then chroma-key it out (my engine tool supports this). If the motion stays inside the silhouette, this works surprisingly well. You can do this in almost any video editing software too! The catch: keying almost always leaves an “aura” (edge spill). My best results come from interpolating the keyed animation with a clean base sprite, so you keep crisp edges and only “borrow” motion/detail where needed. If the animation extends outside the silhouette (tree branches, hair wisps, foliage), I usually skip “true sprite animation” and do it with shaders instead. Keying can’t fully remove halos there, no matter how much feathering/tuning you do. Another annoying issue: pixel corruption. AI rarely generates a perfectly flat background (pure #000000 or #FF00FF). That tiny noise breaks clean extraction and creates crawling garbage pixels around the subject. For clean base sprites (and even PBR maps), a useful trick is generating the same asset on white + black backgrounds and deriving alpha from the difference. This is basically a matte workflow: white = opaque, black = transparent. It fixes aura… but you’d need it per-frame to fix animation, which is still hard. For simple pixel art (single-digit frames), you can sometimes generate a sprite sheet, then ask the model to recreate it on black/white while preserving alignment… but it’s still manual-heavy. Honestly, at this point, for some projects it’s easier to go 3D → 2D and render clean sprites/maps directly. But I still love pushing “pure 2D” and seeing how far we can take it. Thanks for reading! Follow/bookmark/repost if interested in this kind of content!
Startracker 🔺19,493 views • 4 months ago
Big progress made today. Finally fixed cloud shaders and optimized it to not eat VRAM and run smoothly in the browser. There's 2 big shaders involved here - the cloud and the wind which can be programmed to affect all other weather effects. The idea is to have a complete weather system where you can mix and match anything. Done with Cursor with 70% of the work by gpt-codex-xhigh and around 30% by Opus 4.5. Opus rarely oneshots anything and is a total token waster but sometimes it gives a very nice solution. GPT is reliable and precise. And slow. Gets stuff done. Perfect to let it work while going out/eating/sleeping.
Startracker 🔺17,742 views • 5 months ago
![Day 24 of creating a custom game engine using Cursor This time, I created yet another tool to finally finish the unit sprite pipeline - it lets us ingest the 4 maps and the tool automatically cuts and aligns them, ready for animation processing which will then be further processed by the tool I showcased yesterday. I also added the fog of war, basic movement logic and a separate instance server that will form the core gameplay. Inside each tile is a unique instance. It will be populated by a sort of a micro living world on its own. NPCs, enemies, a place to build own outpost - many things will be possible. And I will use this opportunity to talk more about the vision: Imagine finding a bunker in the wasteland, you can explore it, perhaps go underground, find loot or not end up so lucky. Interact with NPCs, learn about the world, the story. And then you follow this story which leads you to other players, other areas. At first glance it would seem like a roguelike procedurally generated world with repetitive quests and gameplay. But in reality, I envision a living world - with a clear goal, that you might never want to achieve because the other activities might be more fun! My vision for the game is to eventually give it endless replay value. With millions of tiles to explore, alone, or with others, you will never get bored. All orchestrated by the [REDACTED]. I played games for 34 years. I have been writing documents/ideas about this project for the past several years. The documentation for this numbers over a hundred pages of mechanics, gameplay and how it all comes together. I will create something no one has made yet - wrapped nicely in the isometric adventure style I adore from Infinity era / Fallout. What genre is it you ask? It's gonna be a mix. Will it all work together? I don't know. But I'm making a game I personally want to get addicted to. And I don't get addicted to games unless they're really really interesting. And lastly, all the tools I make, the engine I make, the AI integrations I make, I hope they'll make other people happy too when they're ready!](https://image.24vids.com/tw-2013704077801394480/amplify_video_thumb/2013703662866980865/img/MON1mPkvaP6Dreor.jpg)
Day 24 of creating a custom game engine using Cursor This time, I created yet another tool to finally finish the unit sprite pipeline - it lets us ingest the 4 maps and the tool automatically cuts and aligns them, ready for animation processing which will then be further processed by the tool I showcased yesterday. I also added the fog of war, basic movement logic and a separate instance server that will form the core gameplay. Inside each tile is a unique instance. It will be populated by a sort of a micro living world on its own. NPCs, enemies, a place to build own outpost - many things will be possible. And I will use this opportunity to talk more about the vision: Imagine finding a bunker in the wasteland, you can explore it, perhaps go underground, find loot or not end up so lucky. Interact with NPCs, learn about the world, the story. And then you follow this story which leads you to other players, other areas. At first glance it would seem like a roguelike procedurally generated world with repetitive quests and gameplay. But in reality, I envision a living world - with a clear goal, that you might never want to achieve because the other activities might be more fun! My vision for the game is to eventually give it endless replay value. With millions of tiles to explore, alone, or with others, you will never get bored. All orchestrated by the [REDACTED]. I played games for 34 years. I have been writing documents/ideas about this project for the past several years. The documentation for this numbers over a hundred pages of mechanics, gameplay and how it all comes together. I will create something no one has made yet - wrapped nicely in the isometric adventure style I adore from Infinity era / Fallout. What genre is it you ask? It's gonna be a mix. Will it all work together? I don't know. But I'm making a game I personally want to get addicted to. And I don't get addicted to games unless they're really really interesting. And lastly, all the tools I make, the engine I make, the AI integrations I make, I hope they'll make other people happy too when they're ready!
Startracker 🔺13,807 views • 4 months ago
No more content to load