It turns out Jev can play Mario Kart 64 really well! Astra computer use has nothing on this

How did you encode the positions and obstacles?

it's reading track position from game memory and feeding it as structured JSON image input would be ideal, but it's not supported yet!

smart and simple, i thought you have some image dithering in place for some elements (im trying to give it vllm abilities 🫠) and let it decide

can Jev play GTA 5?

That’s the frontier

Isn’t Jev frontier? 🤯

@ jev, please crank 90s

This is sick damn

🙏

How are you doing this though I spent two hours trying to do a similar thing and I couldn’t get it to do anything reliable at all. I tried like 7 different ways of converting the screen into text for it to process and all of them sucked. I saw the doom thing they made uses stuff like actual coordinates for where the monsters are and it needs to turn to. That seems like cheating to me or at the very least seems not exciting and useful because it’s not applicable to anything else at all.

I had the model read in-game memory directly instead of trying to use a vision model! totally fair that it feels like "cheating" - it was meant to model the approach that previous genetic algorithm ROM hacks did in the past:

artificial mario intelligence

exactly

Tell Astra to remove Jev and it's going to be even faster, cheaper and more accurate.

in 5-10 years…

GLiNER does computer/browser use just like Jev while being local, free and open weights. it's so lightweight it'll run on a laptop