2 papers
cs.LG2025
BRIDGE: Budget-aware Reasoning via Intermediate Distillation with Guided Examples
Xuan-An Le, Minh-Nam Tran, Son Nguyen
Distilling knowledge from large proprietary models (e.g., GPT-4) to tiny deployable models (less than 1B parameters) faces a critical capacity-budget trap: the 1000x capacity gap b…
cs.LG2025
Improving Adaptive Moment Optimization via Preconditioner Diagonalization
Son Nguyen, Bo Liu, Lizhang Chen +1
Modern adaptive optimization methods, such as Adam and its variants, have emerged as the most widely used tools in deep learning over recent years. These algorithms offer automatic…