Learning What to Imitate: Entropy-Aware Distribution Mixing

Juan Garcia Giraldo, Matteo Santelmo, Eduard Durech +3

Small language models are often post-trained as students on reasoning traces from stronger teacher models to efficiently learn new skills. However, token-level imitation on traces…