26 citations · 48 across the 10 of their papers we have counts for
6 papers · 1 filter
Curriculum Temperature for Knowledge Distillation
Zheng Li, Xiang Li, Lingfeng Yang +5
Most existing distillation methods ignore the flexible role of the temperature in the loss function and fix it as a hyper-parameter that can be decided by an inefficient grid searc…
Efficient One Pass Self-distillation with Zipf's Label Smoothing
Jiajun Liang, Linze Li, Zhaodong Bing +4
Self-distillation exploits non-uniform soft supervision from itself during training and improves performance without any runtime cost. However, the overhead during training is ofte…
RecursiveMix: Mixed Learning with History
Lingfeng Yang, Xiang Li, Borui Zhao +2
Mix-based augmentation has been proven fundamental to the generalization of deep vision models. However, current augmentations only mix samples at the current data batch during tra…
Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Information
Lingfeng Yang, Xiang Li, Renjie Song +5
Fine-grained image classification is a challenging computer vision task where various species share similar visual appearances, resulting in misclassification if merely based on vi…
Decoupled Knowledge Distillation
Borui Zhao, Quan Cui, Renjie Song +2
State-of-the-art distillation methods are mainly based on distilling deep features from intermediate layers, while the significance of logit distillation is greatly overlooked. To…
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective
Quan Cui, Bingchen Zhao, Zhao-Min Chen +5
This work simultaneously considers the discriminability and transferability properties of deep representations in the typical supervised learning task, i.e., image classification.…