← Thursday, October 1, 2026InferencePaper

WUSH-KV uses data-adaptive transforms for low-bit KV-cache quantization

huggingface.co by Jiale Chen et al. Editor score 8/10

Integrated into SGLang, the method reports that at 2-bit it matches or outperforms the OSCAR transform across evaluated models and downstream tasks. Lower KV-cache cost can extend the context or batch sizes available to inference services.

What: WUSH-KV uses data-adaptive transforms to quantize key and value caches at low bit widths.
Method: Calibration data builds separate key and value transforms; the value transform folds into model weights, while the key transform is applied after RoPE.
Why it matters: Lower cache reconstruction error supports more efficient long-context inference, where KV-cache memory and bandwidth costs grow with context length and batch size.

Calibration data builds separate transforms for keys and values before their distinct model paths. Sketched from the article's own text.

Read on huggingface.co →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.