SGLang's banner

SGLang

@sgl_project • 12,501 subscribers

Run LLMs fast at any scale 🔗 https://t.co/F3u6wYESL0 Join our community https://t.co/fmlOfTOEec For AI tech blogs & deep-dives 👉 @lmsysorg

Shorts

Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and the systems behind them. Featuring Lip-Bu Tan (Intel), Lilian Weng (OpenAI), Soumith Chintala (Thinking Machines), Aravind Srinivas (Perplexity) and researchers and builders from across the AI ecosystem. Plus hands-on workshops on model training and serving decision models, hardware Expo, and plenty of community events. What does AI infra mean to you?

Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and the systems behind them. Featuring Lip-Bu Tan (Intel), Lilian Weng (OpenAI), Soumith Chintala (Thinking Machines), Aravind Srinivas (Perplexity) and researchers and builders from across the AI ecosystem. Plus hands-on workshops on model training and serving decision models, hardware Expo, and plenty of community events. What does AI infra mean to you?

20,691 views

SGLang now has Day 0 support for MiniMax-Music 3 from MiniMax (official) 🎵 A music generation model powered by a Qwen3 + RVQ autoregressive backbone and a flow-matching DIT/DAV decoder. POV: you’re parkouring through old Shanghai alleys while Music 3 scores your main-character moment 🎵🧡 Visuals by MiniMax Design (H3), music by MiniMax (official) MiniMax-Music 3. Make your own with SGLang Omni Cookbook👇

SGLang now has Day 0 support for MiniMax-Music 3 from MiniMax (official) 🎵 A music generation model powered by a Qwen3 + RVQ autoregressive backbone and a flow-matching DIT/DAV decoder. POV: you’re parkouring through old Shanghai alleys while Music 3 scores your main-character moment 🎵🧡 Visuals by MiniMax Design (H3), music by MiniMax (official) MiniMax-Music 3. Make your own with SGLang Omni Cookbook👇

14,131 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Muse Spark Open is built to plan, act, and use computer across agentic tasks for life and work. You can now build a powerful personal AI agent with an open stack, and SGLang to make it: - Ultra-responsive with NVFP4 MegaMoE and sequence parallelism - Faster for agentic and coding tasks with our DFlash2 draft model - Customizable for specialized use cases via fine-tuning with Miles We’re incredibly proud to have made this possible with AI at Meta, NVIDIA AI, AI at AMD, Intel, and RadixArk!

Muse Spark Open is built to plan, act, and use computer across agentic tasks for life and work. You can now build a powerful personal AI agent with an open stack, and SGLang to make it: - Ultra-responsive with NVFP4 MegaMoE and sequence parallelism - Faster for agentic and coding tasks with our DFlash2 draft model - Customizable for specialized use cases via fine-tuning with Miles We’re incredibly proud to have made this possible with AI at Meta, NVIDIA AI, AI at AMD, Intel, and RadixArk!

286,622 views • 15 days ago

We turned Qwen3.8-27B into a multimodal decision model. It beat Pokémon FireRed’s elite four and champion with sub-100 ms decisions from live game state. With SGLang’s native /v1/decisions, you can now turn LLMs and VLMs into classification and scoring models. We also added /v1/systemone so Jev-like open models can work with the TypeSafe SDK.

We turned Qwen3.8-27B into a multimodal decision model. It beat Pokémon FireRed’s elite four and champion with sub-100 ms decisions from live game state. With SGLang’s native /v1/decisions, you can now turn LLMs and VLMs into classification and scoring models. We also added /v1/systemone so Jev-like open models can work with the TypeSafe SDK.

253,209 views • 1 month ago

SGLang-Diffusion with VDN-H3 now generates 14.4s of 768p video in just 9.0s 🚀 On 8× B200, 8 step denoising takes just 6.9s, reaching over 2× real time. The 9.0s figure covers the full generation request after warmup. No measured quality regression versus dense 50-step H3 across 103 test prompts. 🧵

SGLang-Diffusion with VDN-H3 now generates 14.4s of 768p video in just 9.0s 🚀 On 8× B200, 8 step denoising takes just 6.9s, reaching over 2× real time. The 9.0s figure covers the full generation request after warmup. No measured quality regression versus dense 50-step H3 across 103 test prompts. 🧵

80,932 views • 1 month ago

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

GLM-5.3 weights from Z.ai are live, with SGLang powering day-0 serving support! GLM-5.3 inherits every optimization and feature we battle-tested for GLM-5.2 over the past months. On real-world multi-turn agentic workloads, we measured 537.6 tok/s/user on NVFP4 and 413 tok/s/user on FP8, at BS=1 with TP8 on 8x B300. It's fast, efficient, and production-ready today on NVIDIA AI Blackwell and Hopper, and AI at AMD MI300X/325X/355X. We believe this is a big step forward for GLM-Series in agentic tasks, with a path toward mythos-class cyber capability. Can't wait to see what people build with GLM-5.3 and SGLang🚀 Cookbook👇

99,455 views • 2 months ago

📢 Qwen's Qwen3.8-2.4T-A95B weights just dropped, and SGLang already has day-0 support with DSpark. HF: Cookbook: 🖖 What kind of model can help optimize an inference engine? Qwen3.8-2.4T-A95B can take on complex engineering tasks, plan and iterate on its own, and keep going until the job is done. We even asked it to optimize its own serving on SGLang, and it ran unattended for 4.5 hours with verified performance improvements. Thanks to the Qwen team Qwen, NVIDIA AI, and AI at AMD for the close collaboration.

📢 Qwen's Qwen3.8-2.4T-A95B weights just dropped, and SGLang already has day-0 support with DSpark. HF: Cookbook: 🖖 What kind of model can help optimize an inference engine? Qwen3.8-2.4T-A95B can take on complex engineering tasks, plan and iterate on its own, and keep going until the job is done. We even asked it to optimize its own serving on SGLang, and it ran unattended for 4.5 hours with verified performance improvements. Thanks to the Qwen team Qwen, NVIDIA AI, and AI at AMD for the close collaboration.

27,317 views • 2 months ago

No more content to load