Zro ♥ SGLang

Published: August 5, 2026 · Updated: September 18, 2026

Zro contributes performance work upstream to SGLang, because an open PR is only the start: the work matters when it is accepted into the library and usable by everyone. This is a live-updated record of those contributions across the SGLang projects.

Contributions

Status	Contribution	Upstream
IN REVIEW	Row-parallel LatentMoE tail for Kimi-K3 prefill on AMD, avoiding duplicated work across tensor-parallel ranks.	SGLang PR #40000 →
MERGED	Spectrum-based diffusion acceleration for Flux, WanVideo, HunyuanVideo, and StableDiffusion3 paths.	SGLang PR #31491 →
MERGED	Make explicit zero-level host and Python tracing settings work in the SGLang JAX profiler.	SGLang JAX PR #1511 →
MERGED	Fused sparse-MLA prefill for DeepSeek Sparse Attention on SGLang JAX.	SGLang JAX PR #1508 →
MERGED	Packed-ragged sparse-MLA prefill with batching, radix, and chunked prefill.	SGLang JAX PR #1514 →
MERGED	Reshard block-quant scales for the JAX 0.10 explicit-mesh shard-map change.	SGLang JAX PR #1513 →
MERGED	Fix sparse prefill so full skip-offset layers stay sparse.	SGLang JAX PR #1512 →
MERGED	Correct DSA indexer-key cache budgeting and memory reporting for sparse MLA.	SGLang JAX PR #1532 →
IN REVIEW	Block verification for speculative-decoding rejection sampling.	SGLang PR #36516 →
IN REVIEW	MXFP4 MoE execution on AMD gfx942 for Kimi K3 serving.	SGLang PR #35525 →
IN REVIEW	Split-KV verify optimization for Kimi K3 serving.	SGLang PR #35521 →
IN REVIEW	Chunked prefix KV for Kimi K3 serving.	SGLang PR #35522 →
IN REVIEW	Zro MLA attention backend for Kimi K3 serving.	SGLang PR #35523 →
IN REVIEW	MoonMath MLA, an MI300X/CDNA3 MLA attention backend for bf16-query / fp8-KV decode.	SGLang PR #32767 →
IN REVIEW	LiteAttention ROCm, an opt-in AMD MI300X attention backend for diffusion.	SGLang PR #28049 →

Row-parallel LatentMoE tail for Kimi-K3, in review

This opt-in path splits Kimi-K3's latent normalization, up-projection, and residual additions across tensor-parallel ranks, then gathers the results instead of repeating the full tail on every GPU. The PR reports a 1.96× speedup for this tail stage at 19,456 tokens on eight MI325X GPUs, not for end-to-end inference. It is off by default, with a token threshold that keeps small batches on the existing path: view PR #40000 →

Spectrum-based diffusion acceleration, merged

This contribution adds Spectrum-based adaptive step-skipping and feature forecasting to SGLang's diffusion runtime. It reduces denoising latency while preserving output quality, and is now part of SGLang: view merged PR #31491 →

Profiling configuration for SGLang JAX, merged

This contribution makes explicit host_tracer_level=0 and python_tracer_level=0 settings work as intended in the SGLang JAX profiler: view merged PR #1511 →

Sparse MLA prefill for SGLang JAX, merged

This opt-in sparse-prefill path makes DeepSeek Sparse Attention useful during prefill, not only decode: view merged PR #1508 →

Packed-ragged sparse-MLA prefill, merged

This builds on the sparse-MLA work with batching, radix, and chunked prefill support, and is now part of SGLang JAX: view merged PR #1514 →

JAX 0.10 block-quant resharding, merged

This fixes block-quant scale resharding for JAX 0.10's explicit-mesh shard-map behavior: view merged PR #1513 →

Sparse-prefill correction, merged

This fixes the sparse-prefill issue we found in Google's code, so full skip-offset layers remain sparse and is now part of SGLang JAX: view merged PR #1512 →

DSA indexer cache accounting, merged

This corrects sparse-MLA KV-pool sizing and memory reporting so DSA indexer-key buffers are budgeted alongside the latent cache. It returns the uncounted allocation from the activation reserve, which matters at long context, and is now part of SGLang JAX: view merged PR #1532 →

Block verification for speculative decoding, in review

This opt-in rejection-sampling path jointly verifies drafted token blocks, which can accept more tokens while preserving the target model's sampling distribution: view PR #36516 →

MXFP4 MoE on gfx942, in review

This adds the MXFP4 mixture-of-experts execution needed for Kimi K3 serving on AMD gfx942 hardware, including MI325X: view PR #35525 →

Split-KV verify optimization, in review

This optimizes the verification path by splitting KV-cache work more efficiently across the GPU: view PR #35521 →

Chunked prefix KV, in review

This adds chunked prefix-KV handling for efficient long-context Kimi K3 serving: view PR #35522 →

Zro MLA attention backend, in review

This adds Zro's MLA attention backend for the Kimi K3 serving stack on AMD CDNA3: view PR #35523 →

MLA decode for Kimi-K2.7-Code on MI300X, in review

moonmath_mla routes the MI300X shape we run to a device-driven paged A16W8 kernel, while all other shapes fall back to aiter. Read the full Kimi-K2.7 MLA benchmark post →, or follow the upstream integration here: view PR #32767 →

LiteAttention-ROCM for diffusion, in review

This opt-in backend brings MoonMath's bf16 forward-attention kernel to SGLang's diffusion runtime on AMD MI300X. It is intentionally scoped to non-causal diffusion attention: view PR #28049 →