← Monday, September 28, 2026Models & trainingBenchmark

NVIDIA Transformer Engine MoE recipe reaches 2.21× Hugging Face throughput

developer.nvidia.com by Michelle Horton 7 min read Editor score 8/10

Grouped expert GEMMs, MXFP8, and a fused MLP kernel delivered up to 2.21× the Hugging Face baseline on eight B200 GPUs. These techniques show how kernel fusion and lower-precision compute can improve MoE training throughput.

What: - NVIDIA's BioNeMo recipe applies Transformer Engine techniques to train mixture-of-experts biological foundation models.
How: - GroupedLinear combines expert GEMMs; MXFP8 reduces memory use, and a fused GroupedMLP kernel avoids intermediate materialization.
Why it matters: - On eight NVIDIA B200 GPUs, the recipe reached up to 2.21× the throughput of the Hugging Face baseline, showing the impact of kernel and precision choices.

Transformer Engine fuses expert computation, activation, and routing-weight scaling in a GroupedMLP kernel. Sketched from the article's own text.

Read on developer.nvidia.com →

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.