96 citations · 233 across the 23 of their papers we have counts for
8 papers · 1 filter
LightViT: Towards Light-Weight Convolution-Free Vision Transformers
Tao Huang, Lang Huang, Shan You +3
Vision transformers (ViTs) are usually considered to be less light-weight than convolutional neural networks (CNNs) due to the lack of inductive bias. Recent works thus resort to c…
Masked Distillation with Receptive Tokens
Tao Huang, Yuan Zhang, Shan You +4
Distilling from the feature maps can be fairly effective for dense prediction tasks since both the feature discriminability and localization priors can be well transferred. However…
Knowledge Distillation from A Stronger Teacher
Tao Huang, Shan You, Fei Wang +2
Unlike existing knowledge distillation methods focus on the baseline settings, where the teacher models and training strategies are not that strong and competing as state-of-the-ar…
Searching for Network Width with Bilaterally Coupled Network
Xiu Su, Shan You, Jiyang Xie +4
Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constrai…
DyRep: Bootstrapping Training with Dynamic Re-parameterization
Tao Huang, Shan You, Bohan Zhang +4
Structural re-parameterization (Rep) methods achieve noticeable improvements on simple VGG-style networks. Despite the prevalence, current Rep methods simply re-parameterize all op…
SimMatch: Semi-supervised Learning with Similarity Matching
Mingkai Zheng, Shan You, Lang Huang +3
Learning with few labeled data has been a longstanding problem in the computer vision and machine learning research community. In this paper, we introduced a new semi-supervised le…
Weak Augmentation Guided Relational Self-Supervised Learning
Mingkai Zheng, Shan You, Fei Wang +4
Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most m…
Relational Surrogate Loss Learning
Tao Huang, Zekang Li, Hua Lu +6
Evaluation metrics in machine learning are often hardly taken as loss functions, as they could be non-differentiable and non-decomposable, e.g., average precision and F1 score. Thi…