ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language
arXiv:2005.07327
Abstract
Person search by natural language aims at retrieving a specific person in a large-scale image pool that matches the given textual descriptions. While most of the current methods treat the task as a holistic visual and textual feature matching one, we approach it from an attribute-aligning perspective that allows grounding specific attribute phrases to the corresponding visual regions. We achieve success as well as the performance boosting by a robust feature learning that the referred identity can be accurately bundled by multiple attribute visual cues. To be concrete, our Visual-Textual Attribute Alignment model (dubbed as ViTAA) learns to disentangle the feature space of a person into subspaces corresponding to attributes using a light auxiliary attribute segmentation computing branch. It then aligns these visual features with the textual attributes parsed from the sentences by using a novel contrastive learning loss. Upon that, we validate our ViTAA framework through extensive experiments on tasks of person search by natural language and by attribute-phrase queries, on which our system achieves state-of-the-art performances. Code will be publicly available upon publication.
arXiv commentECCV2020, 18 pages, 6 figures
References in corpus (12)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
- VQA: Visual Question Answering
- Improving Person Re-identification by Attribute and Identity Learning
- Deep Human Parsing with Active Template Regression
- Ten Years of Pedestrian Detection, What Have We Learned?
- How Far are We from Solving Pedestrian Detection?
- Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search
- Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
- Weakly Supervised Attention Learning for Textual Phrases Grounding
- Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
- Resisting Crowd Occlusion and Hard Negatives for Pedestrian Detection in the Wild