Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Diversity-Aware Reverse Kullback-Leibler Divergence for Large Language Model Distillation
Hoang-Chau Luong, Dat Ba Tran, Lingwei Chen
Reverse Kullback-Leibler (RKL) divergence has recently emerged as the preferred objective for large language model (LLM) distillation, consistently outperforming forward KL (FKL),…
cs.LG2026
Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting
Hoang-Chau Luong, Quang-Thuc Nguyen, Dat Ba Tran +1
Sharpness-Aware Minimization (SAM) was introduced to improve generalization by seeking flat minima, yet it also exhibits robustness to label noise, a phenomenon that remains only p…