← Thursday, October 8, 2026InferencePaper

STEPQuant preserves recurrent-state accuracy at a nominal six-bit budget

huggingface.co by Bingchen Yao et al. Editor score 8/10

STEPQuant assigns precision by error magnitude and memory lifetime, and fits scales to key-row and value-column effects; tests on Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct report accuracy close to FP32 at a nominal 6-bit budget. This targets recurrent-state memory costs in concurrent serving.

What

- STEPQuant is a post-training quantization framework for Delta-rule recurrent states in linear-attention models.

Method

- It allocates precision by error magnitude and memory lifetime, fitting key-row and value-column scales to state distributions and output impact.

Why it matters

- At nominal 6-bit, it closely matches FP32-state accuracy and compresses recurrent state over 5x, reducing serving memory by up to 68.7% in SGLang.

Read on huggingface.co → Code · 83 stars

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.