← Tuesday, October 20, 2026InferenceDeep dive

Jane Street and NVIDIA optimize three Mamba-2 forward-pass kernels with cuTile

developer.nvidia.com by Tanya Lenz 14 min read Editor score 7/10

The implementation uses cuTile for three Mamba-2 kernels, including Tensor Core matrix multiplication to advance hidden state across sequence chunks. The kernel design offers practical material for engineers optimizing GPU execution of state-space models.

What: - Jane Street and NVIDIA optimized three cuTile kernels for the Mamba-2 forward pass: chunk_cumsum, fwd_pass_state, and compute_out.
How: - chunk_cumsum computes decay sums, fwd_pass_state advances hidden state across chunks, and compute_out fuses state, masked interaction, and skip-connection terms.
Why it matters: - Tiled Tensor Core work, register-resident intermediates, and shape-specific autotuning target efficient GPU execution.

The three custom kernels carry computations through the Mamba-2 forward pass. Sketched from the article's own text.

Read on developer.nvidia.com →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.