3 papers
cs.LG2026
Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization
Anh B. H. Nguyen, Ba Tho Phan, Viet Cuong Ta
Knowledge distillation transfers knowledge from a high capacity teacher to a compact student using a mixture of hard and soft losses. On imbalanced data, a fixed weighting between…
cs.LG2025
Saving for the future: Enhancing generalization via partial logic regularization
Zhaorui Tan, Yijie Hu, Xi Yang +3
Generalization remains a significant challenge in visual classification tasks, particularly in handling unknown classes in real-world applications. Existing research focuses on the…
cs.CV2025
Exploiting Layer Normalization Fine-tuning in Visual Transformer Foundation Models for Classification
Zhaorui Tan, Tan Pan, Kaizhu Huang +8
LayerNorm is pivotal in Vision Transformers (ViTs), yet its fine-tuning dynamics under data scarcity and domain shifts remain underexplored. This paper shows that shifts in LayerNo…