← Saturday, October 3, 2026InferenceDeep dive

Helion’s vLLM linear backend improves throughput with per-shape kernel autotuning

pytorch.org by Sean Chen (Red Hat) and Shangdi Yu (PyTorch, Meta Platforms) 12 min read Editor score 8/10

The backend uses one Helion GEMM implementation with per-shape autotuning and hybrid dispatch across algorithm variants; on Hopper GPUs, it outperformed the evaluated CUTLASS and DeepGEMM backends, with over 10% throughput gains on some workloads. This shows how a higher-level kernel DSL can improve inference performance while reducing implementation complexity.

What: Helion adds an autotuned kernel DSL backend to vLLM’s linear operations, using one GEMM implementation across algorithm variants.
How: Per-shape tuning selects among Standard GEMM, Split-K, and Swap-AB; hybrid dispatch beat default backends on evaluated Hopper workloads.
Why it matters: A single high-level kernel with workload-specific tuning can reduce implementation complexity while improving inference throughput.

vLLM’s linear backend uses Helion GEMM, with the autotuner selecting a configuration. Sketched from the article's own text.

Read on pytorch.org →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.