PufferLib 3.0: We trained reinforcement learning agents on 1 Petabyte / 12,000 years of data with 1 server. Now you can, too! Our latest release includes algorithmic breakthroughs, massively faster training, and 10 new environments. Live demos on our site. Volume on for trailer!

Speed and Simplicity: Our latest benchmarks train at up to 5M steps/second on a single RTX 4090. The core training code is 1 file, ~1000 lines. All our code is free and open source at Star to support the project!

New Environments: 10+ new ultra-high perf simulations. Playable on the website, included in PufferLib to accelerate research and prototyping. Many of the simpler ones train 3-4 million steps/second. The more complex ones with larger models like NMMO3 still hit 500k+ per GPU

Hyperparameter Tuning that Works: Protein is our brand new sweep algorithm. It's a heavily modified version of ImbueAI's CARBS that has set SOTA out-of-the-box for multiple clients. Blog post later this week!

Algorithmic Breakthroughs: Until now, we were just making existing methods fast. This release, we've found some truly general improvements to PPO that pair well with Protein. Our new trainer solves problems out of the box that we couldn't solve in 2.0. Blog post later this week!

PufferEnv C API: We're making it easier than ever to build your own high-perf RL environments. You write arbitrary code without a restrictive DSL (looking at you Jax). Tutorial for this env on the website!

All our tools are free and open source, but you can purchase priority service to get our eyes on your problem from $10k/month. We also offer fixed-deliverables terms for larger problems. DM here or email [email protected]

Wow, congrats on the release! Love how you're making RL scalable and accesible

Let's go! PufferLib 3.0 is a game changer for anyone in RL.

LFG ⚡ go go pufferlib 🐡

Huge!! 🔥