2 papers
cs.CV2025
Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head
Penghui Yang, Chen-Chen Zong, Sheng-Jun Huang +2
Traditional knowledge distillation focuses on aligning the student's predicted probabilities with both ground-truth labels and the teacher's predicted probabilities. However, the t…
cs.LG2024
Improving Generalization of Deep Neural Networks by Optimum Shifting
Yuyan Zhou, Ye Li, Lei Feng +1
Recent studies showed that the generalization of neural networks is correlated with the sharpness of the loss landscape, and flat minima suggests a better generalization ability th…