🎉 Reflection-Beam-1.0 from Reflection has Day-0 support in vLLM. 501B total parameters, 23B active per token, and a 1M-token context window. Beam combines hybrid attention with Harmony-format reasoning and tool use. Beam also has Day-0 RL support in SkyRL (NovaSky): LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM... rollouts. We worked with Reflection and Inferact on the serving path, from fused kernels to DFlash. Here is what is under the hood. 🧵show more

Inside Beam: 8 of 256 experts are active per token across 52 layers. Attention alternates a 512-token sliding window with global NoPE attention, pairing local processing with access to long context. Harmony carries the reasoning and tool-use structure. A separate 5-layer drafter supplies speculative tokens.

Long-context serving needs more than a model loader. Inferact's work includes fused all-reduce + norm, packed QKV paths and a tanh softcap kernel. In GB200 forward microbenchmarks, TP4 batch 1 fell from 4.23 to 3.89 ms; TP8 batch 128 from 8.15 to 6.47 ms. These are forward microbenchmarks with dummy weights, not end-to-end serving results. The slide separates kernel timings from forward timings.

DFlash2 speeds up Beam's decode path. Reflection reports 1.72–3.03× decode-only speedups across GSM8K, MATH500, HumanEval, MBPP and MT-Bench at concurrency 1, 16 and 32. GSM8K reaches 3.03× / 2.51× / 2.75×, respectively. Measured in a disaggregated prefill/decode setup; prefill time is excluded. Separately, Reflection reports better throughput with K=7, even though the drafter was trained with block size 16.

🎉 Day-0 RL support for Beam in SkyRL @NovaSkyAI. LoRA, BF16 full fine-tuning and MXFP8 RL, with vLLM rollouts and Reflection’s Megatron-Bridge implementation. SkyRL reports +62% LoRA decode throughput from right-sizing expert adapters in vLLM: 276 → 447 tok/s/GPU at 32 concurrent requests, PP2×TP8. This is a decode result, not an end-to-end training speedup. R3 works with pipeline parallelism, and FP32 router weights are preserved during weight sync. Thanks to Sumanth Hegde and the SkyRL team!

Ready to serve Beam: Docker image: vllm/vllm-openai:reflection-beam Recipe: Blog: In the chat test at concurrency 16, 4× B300 NVFP4 TP4 delivered 1,769 output tok/s; 8× B300 FP8 TP8 delivered 1,300. These use different GPU counts and precisions, with BF16 KV and variable output lengths.

@reflection_ai @NovaSkyAI Any plans for a single DGX Station recipe with the NVFP4 weights? I run DeepSeek V4.1 Flash with hot experts on the GB300 and the rest on an RTX PRO 6000. Curious if Beam can split the same way.

@reflection_ai @NovaSkyAI Day-0 vLLM support for a 501B model is wild. The gap between weights release and usable inference keeps shrinking — that's the real story.

@reflection_ai @NovaSkyAI for the 501b moe at 1m context on day-0: with hybrid attention, how much of that 1m is actually full attention vs the sparse/linear path? also curious if the skyrl day-0 path reuses the same paged attention layout or if rl rollouts force a different kv packing.

@reflection_ai @NovaSkyAI day-0 on 501B/23B active + 1M ctx is what serving people actually need. hybrid attn still on stock kernels or custom path yet?

@reflection_ai @NovaSkyAI Excellent work, thank you to the @vllm_project team!

@reflection_ai @NovaSkyAI 新模型发布当天就有 Day-0 支持，这种速度就是开源推理生态的护城河：大家用脚投票，模型一出来就能跑起来，比什么发布会都管用。

@reflection_ai @NovaSkyAI The 1M context is the part I want to see tested. How does decode speed hold up near the long end?

@reflection_ai @NovaSkyAI How does the KV cache footprint hold up across that 1M context with the hybrid attention setup?