← Tuesday, October 13, 2026InferenceDeep dive

ReplaySSM speeds hybrid-model decoding with checkpointed state and input buffers

developer.nvidia.com by Michelle Horton 15 min read Editor score 9/10

ReplaySSM stores a checkpoint plus recent inputs, cutting state-memory traffic roughly in half and speeding decoding kernels 1.46–1.80x; speculative throughput rises 1.85–2.10x. The design targets memory traffic, rollback, and precision trade-offs in hybrid-model serving.

What: ReplaySSM represents SSM state as a checkpoint plus a buffer of recent inputs.
How: It computes outputs from the checkpoint and buffer, writing a new checkpoint when the buffer fills.
Why it matters: Separating output computation from state updates targets state traffic, speculative-decoding rollback, and quantization error accumulation.

ReplaySSM computes outputs from a checkpoint and input buffer, then updates the checkpoint when the buffer fills. Sketched from the article's own text.

Read on developer.nvidia.com →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.