BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

Bowen Yu, Sheng Zhang, Binhao Wang +8

Chain-of-Thought (CoT) reasoning has significantly improved LLMs' mathematical problem-solving capabilities, but distilling such capabilities into smaller models remains challengin…