Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?
Under review, ACL ARR · arXiv, October 2026
We study OPD from a knowledge-distillation perspective: which teacher corrections help a student reason better? By changing tokens while keeping model weights fixed, we find that the teacher's advantage accumulates across many corrections, and that choosing where to intervene matters.