← Tuesday, October 6, 2026Models & trainingPaper

Fixed-point training improves looped models and speeds prefill and RL updates

huggingface.co by Benhao Huang et al. Editor score 9/10

The work uses learned depth priors and orthogonal input injection to improve perplexity across 100M–1.6B models, and reports 1.79× faster student prefill and 2× faster RL updates. Its fixed-point analysis connects model architecture choices to training and inference costs.

What

Fixed-point training tunes looped language models to reduce recurrence costs across training, decoding, prefill, and reinforcement learning.

Method

A learned depth prior and orthogonal input injection lower perplexity across tested scales; fixed points enable truncated backpropagation and terminal KV sharing.

Why it matters

For model builders, the approach enables KV sharing with almost no accuracy loss, prefill up to 1.79× faster, and RL gradient updates 2× faster.

Read on huggingface.co → Code · 29 stars

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.