Cross-modality Person re-identification with Shared-Specific Feature Transfer

Abstract

Cross-modality person re-identification (cm-ReID) is a challenging but key technology for intelligent video analysis. Existing works mainly focus on learning common representation by embedding different modalities into a same feature space. However, only learning the common characteristics means great information loss, lowering the upper bound of feature distinctiveness. In this paper, we tackle the above limitation by proposing a novel cross-modality shared-specific feature transfer algorithm (termed cm-SSFT) to explore the potential of both the modality-shared information and the modality-specific characteristics to boost the re-identification performance. We model the affinities of different modality samples according to the shared features and then transfer both shared and specific features among and across modalities. We also propose a complementary feature learning strategy including modality adaption, project adversarial learning and reconstruction enhancement to learn discriminative and complementary shared and specific features of each modality, respectively. The entire cm-SSFT algorithm can be trained in an end-to-end manner. We conducted comprehensive experiments to validate the superiority of the overall algorithm and the effectiveness of each component. The proposed algorithm significantly outperforms state-of-the-arts by 22.5% and 19.3% mAP on the two mainstream benchmark datasets SYSU-MM01 and RegDB, respectively.

arXiv commentTo appear at CVPR2020

References in corpus (10)

• In Defense of the Triplet Loss for Person Re-Identification
• Person Re-identification: Past, Present and Future
• Random Erasing Data Augmentation
• MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval
• Robust Depth-based Person Re-identification
• A New Benchmark and Approach for Fine-grained Cross-media Retrieval
• Attend to the Difference: Cross-Modality Person Re-identification via Contrastive Correlation
• Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
• Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification
• FANet: Quality-Aware Feature Aggregation Network for Robust RGB-T Tracking