← Tuesday, September 29, 2026Models & trainingPaper

Domain-normalized distillation improves multi-teacher results across model sizes

huggingface.co by Xin Li et al. Editor score 8/10

DN-MOPD rescales each specialist's feedback by its measured spread; it improves average scores over standard MOPD at three model sizes, across six benchmarks, three seeds and two answer-length limits. The method addresses how imbalanced teacher feedback can blunt multi-skill model training.

What

Domain-Normalized MOPD rescales feedback from specialist teachers by each domain’s measured spread during multi-teacher on-policy distillation.

Method

Across three Qwen3.5 sizes, it improves average scores over MOPD on six benchmarks and recovers most of the lost mathematics gain.

Why it matters

Balancing domain feedback can help engineers distill several specialist capabilities into one shared model.

Read on huggingface.co → Code · 5 stars

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.