Releasing PufferLib 5.0: Train agents in under a second

5 Exciting new results brought to you by our team + contributors: @spenccheng @finlay_sanders (fintern!) @ValtteriValo @daphnesolves @elliotarledge. Play with the agents at All experimental data available is available in Constellation online + local.

5 levels of parallelism: bitwise deterministic async training in <10,000 lines of CUDA C including Protein hparam sweeps, Constellation vis, GPU/CPU env support, and standalone CPU eval. No more Python.

5 times faster training: that's rough average solve time for the same tasks vs. 4.0 from async, kernel tweaks, and algorithmic improvements. 3x pretty much across the board, some envs 10x. Peak SPS is >60M. Check back later this week for in-depth articles!

We now have ~60 environments in Ocean available directly on Almost all have strong baselines that run right in your browser. Locally, not on our servers. Because you can do that with models this small!

Everything you see here is free and open source. We also build ultra high performance environments and agents for non-LLM use cases professionally and have applied our tech in gaming, animation, finance, commerce, and defense. Contact us to puff up your training!

It's so easy to use it one shotted pretty much every env I ever built. It can even learn to play existing games as long as you get the scaffolding correct. One shot F-Zero X PC Port.

removing all the dependency fluff is amazing

hey i think you dropped this, king 👑

puffed up🔥

Smh, just finished setting up my local workflow with puffer 4.0 and already got to upgrade (Kidding, good job, thanks for new release 🙏)

5.0 is even easier to set up. 4.0 had like 9 python deps + annoying CUDA version restrictions with torch. All gone now

Checks out. Cartpole running on my gpu without importing pytorch

I'm really interested in testing out pufferlib but I'm having a really hard time finding extensive docs or examples, should I be looking somewhere on the site or a github repo?

this is sick

huge!!!

huge W

really nice results and release🔥🔥

Nice man

might need to change your bio now cuz this is insane

Useful for autonomous drones?

legend 🙌🏽

Training an RL agent in under a second completely changes the development cycle. When your feedback loop drops from hours to milliseconds, you stop guessing hyper-parameters and start treating reinforcement learning like standard software compilation.

Can’t wait to get home and test it!

well done!

So fast

ooh i might use this actually for some of the astra/spatial RL stuff i'm looking into

Puffer ftw!

This is awesome! Thanks for your work on this! 🙏

Massive congrats on the release

nice work

try this prompt "show me native pufferlib 5.0 cuda, no python, no torch"