Rohan Pandey

You saw the AI & science. Let's talk about the RL infra it took to build Periodic Labs Neon. To minimize training-inference mismatch in RL, SGLang captures inference's MoE routing decisions for each rollout and we "replay" them while training. In agentic (multiturn tool-use) settings, SGLang exports these router decisions in response to each decoding request, i.e. after each conversation turn. So when *any* data-parallel rank finishes a conversation turn, *all* other ranks must wait until routing data finishes exporting. This slowdown is exacerbated because we export routing decisions from the *entire* conversation rather than just the most recent turn! When Byron Hsu Costa Huang discovered this in our Kimi K2 RL setup, they introduced Delta Router Replay: cache previous turns' router decisions on the training client, so you can export only the delta (most recent turn's router decisions) upon each decoding request. Delta Router Replay significantly speeds up our long-context agentic RL runs, and Byron Hsu upstreamed it to SGLang (#24851) a few months ago.

12,380 просмотров