Loading video...

Video Failed to Load

Go Home

Serving GLM5.2 NVFP4 Agentic Workload with SGLang: How We Reached 500 TPS on 8xB300 at bs=1 In this deep dive, we explain how SGLang reaches 500+ tok/s/user at bs=1 on 8xB300, with 18 to 34% higher single-user interactivity within two weeks since day-0, and 6 to 11% better peak...

104,832 views • 9 days ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos