Look ma, my AI-powered Jax beats Jev and Laya when benchmarked correctly.

Creator of laya here. Super cool

What kind of benchmark setup are you runing here? Never heard of Jax, is it also focused purely on classification workloads?

Jax is my own experimental classifier - Jev api compatible so I can run it locally. Currently I'm benchmarking differently in Python. This is plain frontend Javascript workfloads benchmarking a game - meaningless but it's been shown around on X, pretending Laya is beating Jev

Thank you Tom! Would love to know more once you make more progress. I am looking now in general for reliable classifiers.

reliable is a loose term with llm-based classifiers. but will let you know

Very cool! Thanks for using pink for Jev!

It's your color - I fixed it (it's reversed in the video) lol

Dope experiment mate. What's the algo/concept behind Jax?

Train a small model to classify. It reads input, but generates nothing (no inferrence). Shows raw numbers what it would have predicted for the schema/question/options

congrats!

Is there a known strategy in Snake that suggests you go back to the left wall every couple of seconds? It seems like all 3 models behave that way, even when it was super inefficient (in the early game). Curious what context led to that.

I couldn't tell you why. looks like the training data suggest "left" as the safest bet

ok but where's JAX?

Jax is tugged in and sleeping (still in development). Will be used for code classification and don't think it'll be useful for the general public. (Un)surprisingly albeit trained on code it can still play games

Look ma energy is strong. I’ll take a live walk on my own app over a chart though. Same repo, same flow. That’s the only benchmark I actually trust.

That's just a frontend X-friendly benchmark. I can't look inside Jev tho :( But it's fun to reconstruct it.

Fair. Chart’s for the timeline. The reconstruction’s the actual job.

Can Jev really plan ahead of the moves for the end game? I really wanted to see the final part for each model? I don't think there is an intelligence in the jev. It is just an classifier model.

There is. And no it can't predict very far ahead. You see these tendencies to go "left" and "down-right" zigzag? These are trained words. check jevs chess demo - you'd see without intelligence you can't do much with a simple classifier

Jax is a machine learning library? Is it the same or you named it the same?

what specific benchmarks did you use to compare Jax against Jev and Laya?

proud of u son