AI-generated editorial illustration; not a documentary photograph.
SGLang v0.5.20, released on September 18, 2026, expands the LLM serving framework with replayable sampling masks, a CPU-only simulator, unified prefix caching and new accelerator images. The release incorporates 713 pull requests from 237 contributors and also removes several older runtime and deployment paths.
Replaying RL Rollouts
The new return_sampling_mask option records the exact token set available at every decoding step and the sampled token's log-probability. Trainers can therefore replay reinforcement-learning rollouts without reconstructing top-k or top-p decisions. Sampling masks now work with overlap scheduling and DisallowedTokensLogitsProcessor; capacity defaults to 4,096 tokens and can be changed with –sampling-mask-max-tokens.
SGLang's published Qwen3-8B measurements show the overlap implementation raising decode throughput by 17% at batch size 1 and 52% at batch size 64 compared with its previous mask implementation. These figures are project benchmarks rather than independent test results.
Cache and Simulation Changes
A unified radix tree now preserves sliding-window attention state where requests branch from a shared prefix. In the project's DeepSeek-V4-Flash test with a common system prompt, the token hit rate increased from 43.8% to 60.8%, while mean time to first token fell from 1.57 seconds to 1.07 seconds. An optional external linker can connect the tree to shared memory through Mooncake or UMBP.
The release also introduces SGLang Simulator, which runs the actual scheduler, radix cache and hierarchical cache on CPUs while replacing model execution with a latency predictor. SGLang reports time-to-first-token predictions within roughly 6% for most measured traces, rising to 10% for the longest 32K-to-128K traces, making the tool useful for cache and scheduling experiments that do not require GPUs.
Hardware-Specific Gains
ROCm model loading receives a major optimization by staging large pageable host-to-device transfers. According to the release measurements, GLM-5.2 at tensor parallelism 4 loaded on four MI355X GPUs in 40.4 seconds, down from 505.7 seconds. Intel XPU users now receive a versioned Docker image with each tagged release, alongside new ROCm 10, gfx1151 and Moore Threads MUSA images.
DeepSeek-V4 also gains TRT-LLM attention kernels for Blackwell SM100 and SM103 hardware. SGLang reports about 1.2 times faster prefill and 1.45 times faster decode than FlashMLA at the kernel level on B200. Separate opt-in changes accelerate DeepSeek-V4-Flash on four RTX PRO 6000 GPUs, with batch-one decode TPOT falling from 36.1 milliseconds to 10.5 milliseconds in the project's test.
Deployment Breaks to Check
Responses API storage is now disabled unless the server starts with –enable-response-store. Without that flag, result retrieval, previous_response_id chaining and background requests return HTTP 400; prefill-decode deployments cannot enable the store. The original prefill context-parallel runtime and its command-line options have also been removed, while the replacement path currently rejects HIP, NPU and MUSA. CUDA 12 wheels and images ended with v0.5.19, and ROCm 7.0 artifacts are retired.
Source: SGLang — v0.5.20
SIGNAL TITANS
Leave a Reply