← Monday, September 28, 2026InferenceDeep dive

Confidential inference retains up to 98.2% of baseline throughput on B200 GPUs

developer.nvidia.com by Tanya Lenz 7 min read Editor score 8/10

On eight B200 GPUs running DeepSeek-R1, confidential compute retained 96.1%–98.2% of baseline output-token throughput with 1.2%–4.3% per-token latency overhead. The implementation details show how memory protection affects inference performance and data movement.

What

NVIDIA Confidential Computing enables production inference in trusted hardware using encrypted confidential VMs, confidential GPUs, and encrypted NVLink.

How

TensorRT LLM adapts secure data movement, kernel autotuning, and multi-GPU communication; tests on eight B200 GPUs retained 96.1%–98.2% of baseline output-token throughput.

Why it matters

Frameworks must account for secure memory movement and communication limits to preserve inference performance in confidential deployments.

Read on developer.nvidia.com →

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.