Loading video...
Video Failed to Load
I wrote the full Craftax RL environment + PPO in one CUDA file. zero python in the loop. training from scratch on my RTX PRO 6000 > 0.6s: first wood, first crafting table, first dungeon entered > 1.7s: first zombie + skeleton kills, first bow found > 2.9s: first... show more
51,510 views • 3 months ago •via X (Twitter)
36 Comments
no single magic kernel. it is boring systems work stacked end to end: env (312M steps/s, no NN in the loop): - struct-of-arrays state so every warp load is coalesced - worldgen resets are warp-cooperative: one warp rebuilds a dying world's noise fields in shared memory, bit-exact with the scalar path - rare divergent events (mob spawn scans) get compacted into queues drained by a separate kernel instead of stalling every thread - the whole step is CUDA-graph captured training (5.5M steps/s at the quality recipe, 42M full-batch): - the entire PPO loop lives on the GPU: rollout, GAE, backward, Adam. no python, no transfers, weights never leave the card - obs recorded as 996 compact uint8 bytes, then expansion + encoder run as one big GEMM instead of per-env scatter loops (2.4x on the trainer) - GRU chain backward chunked to fit in L2 - analytic grads checked against finite differences, trajectories hash-sealed against the C reference build i actually tried some ambitious kernel fusion but ended up losing to a plain GEMM and env separation strategy
Incredible work
❤️
8192 is max sps for gb10 (dgx spark) training a policy now
wow!
That’s pretty cool! training on craftax won’t give you an agent that generalizes, but if you can procedurally change the environment and automatically update your cuda code, then you can quickly meta learn something interesting…
Ok can i play Minecraft with claude now or in near future
a tiny policy will beat minecraft for you in a min or so at my achieved sim speed. need to cook for another day
What about a friend maxxing / playmate rl env pls for us lonesome folk
how does the trained policy perform back in the original craftax environment?
the environment count becoming the video beats another terminal chart. 524,288 tiny worlds is hard to argue with.
Bruh how?? Insane!
now this is some next lvl shit
<3
epic, nice visuals too
What mfs will do just to avoid working on sample efficient RL methods
gpu gaming
Impressive
well done
Huge 👀 Great work 👏
Wow, would be cool to see some charts with before/after (original Craftax vs your version)
Hell yeah brother
This is insane
Amazing! I did my Masters in Curoisity-Based RL for managing sparse rewards pre-Craftax. I attempted to build my own env and trained on it, but a few months later they came out with Craftax. This is just next level. Will be checking it out soon!
crazyyy
impressive speed benchmarks on that RTX PRO 6000, 312M env steps/s is wild
Holy aura will be checking this out
This is the way to go!
The claim of zero python in the loop overstates the setup. The environment still relies on python for initialization and orchestration.
This is an insanely amazing project
That's based
This is awesome!
@yacineMTB What a legend
Very cool!
Absolutely insane
is this a sign to learn cuda
