← Sunday, October 18, 2026Models & trainingPaper

SAE features show measurable pairwise interactions in model behavior

huggingface.co by Haotian Ma et al. Editor score 8/10

The paper defines SAE Additivity Residuals and finds lower first-order residuals for learned coordinates across 18 model-behavior settings, while pairwise terms improve held-out coalition ordering. Its measured results caution against assuming sparse features have additive effects when interpreting or steering models.

What

The paper introduces SAE Additivity Residuals to measure behavioral variation unexplained by interactions up to a chosen order.

Method

Learned SAE coordinates had lower first-order residuals than rotated coordinates, but pairwise terms improved held-out coalition ordering in every setting.

Why it matters

Engineers interpreting or steering models may need to account for interactions between features, not just individual feature effects.

Read on huggingface.co → Code · 5 stars

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.