23 citations · 81 across the 9 of their papers we have counts for
9 papers · 1 filter
Effective Whole-body Pose Estimation with Two-stages Distillation
Zhendong Yang, Ailing Zeng, Chun Yuan +1
Whole-body pose estimation localizes the human body, hand, face, and foot keypoints in an image. This task is challenging due to multi-scale body parts, fine-grained localization f…
CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Anyi Rao +3
Recent vision-language models have achieved tremendous advances. However, their computational costs are also escalating dramatically, making model acceleration exceedingly critical…
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels
Zhendong Yang, Ailing Zeng, Zhe Li +3
Knowledge Distillation (KD) uses the teacher's prediction logits as soft labels to guide the student, while self-KD does not need a real teacher to require the soft labels. This wo…
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
Dachuan Shi, Chaofan Tao, Ying Jin +3
Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities. Moreover, increasingly heavier models,…
ViTKD: Practical Guidelines for ViT feature knowledge distillation
Zhendong Yang, Zhe Li, Ailing Zeng +3
Knowledge Distillation (KD) for Convolutional Neural Network (CNN) is extensively studied as a way to boost the performance of a small model. Recently, Vision Transformer (ViT) has…
Rethinking Knowledge Distillation via Cross-Entropy
Zhendong Yang, Zhe Li, Yuan Gong +4
Knowledge Distillation (KD) has developed extensively and boosted various tasks. The classical KD method adds the KD loss to the original cross-entropy (CE) loss. We try to decompo…
Masked Generative Distillation
Zhendong Yang, Zhe Li, Mingqi Shao +3
Knowledge distillation has been applied to various tasks successfully. The current distillation algorithm usually improves students' performance by imitating the output of the teac…
Transparent Shape from a Single View Polarization Image
Mingqi Shao, Chongkun Xia, Zhendong Yang +2
This paper presents a learning-based method for transparent surface estimation from a single view polarization image. Existing shape from polarization(SfP) methods have the difficu…
Focal and Global Knowledge Distillation for Detectors
Zhendong Yang, Zhe Li, Xiaohu Jiang +4
Knowledge distillation has been applied to image classification successfully. However, object detection is much more sophisticated and most knowledge distillation methods have fail…