Learning to Maximize Mutual Information for Chain-of-Thought Distillation

Xin Chen, Hanxian Huang, Yanjun Gao +3

Knowledge distillation, the technique of transferring knowledge from large, complex models to smaller ones, marks a pivotal step towards efficient AI deployment. Distilling Step-by…