I wrote the full Craftax RL environment + PPO in one CUDA file. zero python in the loop. training from scratch on my RTX PRO 6000 > 0.6s: first wood, first crafting table, first dungeon entered > 1.7s: first zombie + skeleton kills, first bow found > 2.9s: first... iron > 14s: first iron sword > 18s: first diamond > 71s: first diamond pickaxe > 16.7 min: first time reaching floor 3 of 9 (gnomish mines) > 75 min: 25,000,000,000 steps done up to 524,288 games running at once. env stepping alone hits 312M steps/s.show more

no single magic kernel. it is boring systems work stacked end to end: env (312M steps/s, no NN in the loop): - struct-of-arrays state so every warp load is coalesced - worldgen resets are warp-cooperative: one warp rebuilds a dying world's noise fields in shared memory, bit-exact with the scalar path - rare divergent events (mob spawn scans) get compacted into queues drained by a separate kernel instead of stalling every thread - the whole step is CUDA-graph captured training (5.5M steps/s at the quality recipe, 42M full-batch): - the entire PPO loop lives on the GPU: rollout, GAE, backward, Adam. no python, no transfers, weights never leave the card - obs recorded as 996 compact uint8 bytes, then expansion + encoder run as one big GEMM instead of per-env scatter loops (2.4x on the trainer) - GRU chain backward chunked to fit in L2 - analytic grads checked against finite differences, trajectories hash-sealed against the C reference build i actually tried some ambitious kernel fusion but ended up losing to a plain GEMM and env separation strategy

Incredible work

❤️

8192 is max sps for gb10 (dgx spark) training a policy now

wow!

That’s pretty cool! training on craftax won’t give you an agent that generalizes, but if you can procedurally change the environment and automatically update your cuda code, then you can quickly meta learn something interesting…

Ok can i play Minecraft with claude now or in near future

a tiny policy will beat minecraft for you in a min or so at my achieved sim speed. need to cook for another day

What about a friend maxxing / playmate rl env pls for us lonesome folk

how does the trained policy perform back in the original craftax environment?

the environment count becoming the video beats another terminal chart. 524,288 tiny worlds is hard to argue with.

Bruh how?? Insane!

now this is some next lvl shit

<3

epic, nice visuals too

What mfs will do just to avoid working on sample efficient RL methods

gpu gaming

Impressive

well done

Huge 👀 Great work 👏

Wow, would be cool to see some charts with before/after (original Craftax vs your version)

Hell yeah brother

This is insane

Amazing! I did my Masters in Curoisity-Based RL for managing sparse rewards pre-Craftax. I attempted to build my own env and trained on it, but a few months later they came out with Craftax. This is just next level. Will be checking it out soon!

crazyyy

impressive speed benchmarks on that RTX PRO 6000, 312M env steps/s is wild

Holy aura will be checking this out

This is the way to go!

The claim of zero python in the loop overstates the setup. The environment still relies on python for initialization and orchestration.

This is an insanely amazing project

That's based

This is awesome!

@yacineMTB What a legend

Very cool!

Absolutely insane

is this a sign to learn cuda