Low-Resolution Object Recognition with Cross-Resolution Relational Contrastive Distillation

Abstract

Recognizing objects in low-resolution images is a challenging task due to the lack of informative details. Recent studies have shown that knowledge distillation approaches can effectively transfer knowledge from a high-resolution teacher model to a low-resolution student model by aligning cross-resolution representations. However, these approaches still face limitations in adapting to the situation where the recognized objects exhibit significant representation discrepancies between training and testing images. In this study, we propose a cross-resolution relational contrastive distillation approach to facilitate low-resolution object recognition. Our approach enables the student model to mimic the behavior of a well-trained teacher model which delivers high accuracy in identifying high-resolution objects. To extract sufficient knowledge, the student learning is supervised with contrastive relational distillation loss, which preserves the similarities in various relational structures in contrastive representation space. In this manner, the capability of recovering missing details of familiar low-resolution objects can be effectively enhanced, leading to a better knowledge transfer. Extensive experiments on low-resolution object classification and low-resolution face recognition clearly demonstrate the effectiveness and adaptability of our approach.

This paper is accepted by IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Representation Learning with Contrastive Predictive Coding
• Knowledge Distillation: A Survey
• FitNets: Hints for Thin Deep Nets
• Learning deep representations by mutual information estimation and maximization
• Learning Face Representation from Scratch
• Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation
• On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques
• Paraphrasing Complex Network: Network Compression via Factor Transfer
• Face Recognition in Low Quality Images: A Survey
• Unconstrained Face Detection and Open-Set Face Recognition Challenge
• Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation