← Monday, October 19, 2026InferencePaper

DLite removes historical KV-cache dependence from speculative decoding

huggingface.co by Jiaming Xu et al. Editor score 8/10

DLite uses target-model hidden states for local speculative prediction instead of maintaining historical context; existing draft models lose up to 80% of acceptance length on agentic workloads. The approach targets both acceptance degradation and KV-cache overhead in multi-turn serving.

What

DLite is a history-free speculative decoding framework designed for agentic workloads with long, evolving contexts and frequent multi-turn interactions.

Method

It reuses target-model contextual hidden states for local speculative prediction, avoiding explicit historical-context maintenance; a sequential head restores intra-block causal dependencies.

Why it matters

Experiments report 2–6 acceptance length and up to 70% improvement over DSpark, while avoiding the overhead of maintaining historical KV cache.

Read on huggingface.co → Code · 13 stars

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.