CMTR: Cross-modality Transformer for Visible-infrared Person Re-identification

Abstract

Visible-infrared cross-modality person re-identification is a challenging ReID task, which aims to retrieve and match the same identity's images between the heterogeneous visible and infrared modalities. Thus, the core of this task is to bridge the huge gap between these two modalities. The existing convolutional neural network-based methods mainly face the problem of insufficient perception of modalities' information, and can not learn good discriminative modality-invariant embeddings for identities, which limits their performance. To solve these problems, we propose a cross-modality transformer-based method (CMTR) for the visible-infrared person re-identification task, which can explicitly mine the information of each modality and generate better discriminative features based on it. Specifically, to capture modalities' characteristics, we design the novel modality embeddings, which are fused with token embeddings to encode modalities' information. Furthermore, to enhance representation of modality embeddings and adjust matching embeddings' distribution, we propose a modality-aware enhancement loss based on the learned modalities' information, reducing intra-class distance and enlarging inter-class distance. To our knowledge, this is the first work of applying transformer network to the cross-modality re-identification task. We implement extensive experiments on the public SYSU-MM01 and RegDB datasets, and our proposed CMTR model's performance significantly surpasses existing outstanding CNN-based methods.

arXiv comment11 pages, 7 figures, 7 tables

References in corpus (9)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Decoupled Weight Decay Regularization
• Learning Discriminative Features with Multiple Granularities for Person Re-Identification
• Person Re-identification: Past, Present and Future
• A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification
• Parameter Sharing Exploration and Hetero-Center based Triplet Loss for Visible-Thermal Person Re-Identification
• Deep Learning for Person Re-identification: A Survey and Outlook
• Illumination-Adaptive Person Re-identification

Cited by in corpus (3)

• Bridging the Gap: Multi-Level Cross-Modality Joint Alignment for Visible-Infrared Person Re-Identification
• Robust Duality Learning for Unsupervised Visible-Infrared Person Re-Identification
• Enhancing Visible-Infrared Person Re-identification with Modality- and Instance-aware Visual Prompt Learning