← Monday, October 5, 2026InferenceRelease
vLLM 0.31.0 adds GPU optimizations and persistent weight caching
The release adds SM100 attention and KV-cache optimizations, fused operations, and a preload CLI that keeps post-quantized weights resident across engine restarts. These changes target inference performance and faster recovery of serving capacity.
- What
- vLLM 0.31.0 is a release with GPU optimizations, serving features, scheduling controls, and fixes.
- How
- A new preload CLI launches a daemon that keeps post-quantized weights in GPU memory across engine restarts.
- Why it matters
- Persistent weight caching can support faster restarts in serving systems, while the release also adds controls for request admission and prefill scheduling.
Discussion
Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.
Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.
