Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification

Abstract

Text-to-image person re-identification (ReID) aims to search for images containing a person of interest using textual descriptions. However, due to the significant modality gap and the large intra-class variance in textual descriptions, text-to-image ReID remains a challenging problem. Accordingly, in this paper, we propose a Semantically Self-Aligned Network (SSAN) to handle the above problems. First, we propose a novel method that automatically extracts semantically aligned part-level features from the two modalities. Second, we design a multi-view non-local network that captures the relationships between body parts, thereby establishing better correspondences between body parts and noun phrases. Third, we introduce a Compound Ranking (CR) loss that makes use of textual descriptions for other images of the same identity to provide extra supervision, thereby effectively reducing the intra-class variance in textual features. Finally, to expedite future research in text-to-image ReID, we build a new database named ICFG-PEDES. Extensive experiments demonstrate that SSAN outperforms state-of-the-art approaches by significant margins. Both the new ICFG-PEDES database and the SSAN code are available at https://github.com/zifyloo/SSAN.

arXiv commentA new database for text-to-image ReID is provided. Code will be released

References in corpus (9)

• VSE++: Improving Visual-Semantic Embeddings with Hard Negatives
• Deep Representation Learning with Part Loss for Person Re-Identification
• Multi-task Learning with Coarse Priors for Robust Part-aware Person Re-identification
• Person Re-identification Meets Image Search
• Identity-Aware Textual-Visual Matching with Latent Co-attention
• Sequence-based Person Attribute Recognition with Joint CTC-Attention Model
• Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification
• Visual-Textual Association with Hardest and Semi-Hard Negative Pairs Mining for Person Search
• Factorized Distillation: Training Holistic Person Re-identification Model by Distilling an Ensemble of Partial ReID Models

Cited by in corpus (3)

• Look Before You Leap: Improving Text-based Person Retrieval by Learning A Consistent Cross-modal Common Manifold
• RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search
• MARS: Paying more attention to visual attributes for text-based person search