← Friday, October 9, 2026ArchitectureDeep dive

NVIDIA Dynamo uses session IDs to route agent inference and move KV cache

pytorch.org by Ishan Dhanani, Jamie Li, Karen Chung 15 min read Editor score 8/10

Dynamo uses a unified session identifier for session-aware routing, shared KV-cache indexing and cache movement across vLLM and SGLang. This program-aware design targets agent workloads with long contexts, parallel subagents and idle time between tool calls.

What: NVIDIA Dynamo adds a unified session-level ID to make inference serving aware of agent sessions and subagents.
How: It uses session-linked tracing, session-aware routing and admission control, shared KV cache indexing, and programmatic cache movement.
Why it matters: Session-aware serving can reuse resident context and manage working sets, addressing cache thrashing in multi-call agent workloads.

Session identifiers help Dynamo route requests and manage KV cache for agent sessions. Sketched from the article's own text.

Read on pytorch.org →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.