SGLang-Diffusion is a high-performance serving framework for diffusion models, designed for both large-scale offline generation and latency-sensitive real-time inference. At PyTorch Conference North America 2026, Yihao Wang and Kevin Mi of RadixArk will present the design of SGLang-Diffusion and the PyTorch-based inference optimizations behind it, including efficient request scheduling,... memory management, batching strategies, and execution-path optimizations for diffusion pipelines. Join us in San Jose, October 20-21: #PyTorchConshow more

Optimizing diffusion serving frameworks with generic LLM dynamic batching strategies sounds phenomenal until multi-step iterative denoising loops trigger severe VRAM memory fragmentation. How does SGLang-Diffusion guarantee predictable tail-latency when cross-attention spatial tensor transformations disrupt the active scheduling queue under high-concurrency production workloads?

@radixark For real-time diffusion, scheduling under bursty, mixed-size requests is central. Reporting p95/p99 latency, throughput, batch size, and quality across concurrency levels would make the offline-versus-live trade-off easier to evaluate.

@radixark How much does this cut per-image serving cost?

@radixark These optimization points are exactly what diffusion inference needs. Looking forward to their sharing at the conference.

the scheduling differences between diffusion and autoregressive serving are what i'd dig into first. diffusion requests keep revisiting the same latent state, so i'd expect memory residency and batching decisions to behave very differently from token decoding. curious how SGLang-Diffusion handles that.

@radixark Diffusion serving finally getting proper attention, love it