SGLang now supports DSpark, enabling confidence-driven, variable-length verification for speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in... SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇show more

Run DeepSeek-V4-Flash with DSpark (on H200, TP4 + DP-attention): SGLANG_RAGGED_VERIFY_MODE=compact python3 -m sglang.launch_server \ --model-path deepseek-ai/DeepSeek-V4-Flash-DSpark \ --speculative-algorithm DSPARK \ --tp 4 --dp-size 4 --enable-dp-attention --enable-dp-lm-head \ --moe-runner-backend flashinfer_mxfp4 \ --swa-full-tokens-ratio 0.1 --chunked-prefill-size 1024 \ --mem-fraction-static 0.9 --cuda-graph-max-bs 192 \ --trust-remote-code --host 0.0.0.0 --port 30000

Run DeepSeek-V4-Pro with DSpark (on B300, TP8): SGLANG_RAGGED_VERIFY_MODE=compact python3 -m sglang.launch_server \ --model-path deepseek-ai/DeepSeek-V4-Pro-DSpark \ --speculative-algorithm DSPARK \ --tp 8 \ --moe-runner-backend flashinfer_mxfp4 \ --mem-fraction-static 0.82 --cuda-graph-max-bs 192 --chunked-prefill-size 4096 \ --trust-remote-code --host 0.0.0.0 --port 30000

Full technical details, benchmarks, and more configs in the blog 🫡

🙌

Is it available upstream? Didnt see a merged related commit.

dspark helps, but tail latency still spikes when draft confidence drifts under real traffic

interesting, does this mean higher throughput under load without sacrificing quality

DSpark 335 tok/s. MTP 213. Oh wait... I don't trust B=1 benchmarks. DSpark holds at B=256. MTP falls off. That's production-grade.

In a Nvidia Spark? 👀