Person Search with Natural Language Description
arXiv:1702.05729
Abstract
Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance. Existing methods mainly focused on searching persons with image-based or attribute-based queries, which have major limitations for a practical usage. In this paper, we study the problem of person search with natural language description. Given the textual description of a person, the algorithm of the person search is required to rank all the samples in the person database then retrieve the most relevant sample corresponding to the queried description. Since there is no person dataset or benchmark with textual description available, we collect a large-scale person description dataset with detailed natural language annotations and person samples from various sources, termed as CUHK Person Description Dataset (CUHK-PEDES). A wide range of possible models and baselines have been evaluated and compared on the person search benchmark. An Recurrent Neural Network with Gated Neural Attention mechanism (GNA-RNN) is proposed to establish the state-of-the art performance on person search.
References in corpus (16)
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Deep Residual Learning for Image Recognition
- Microsoft COCO Captions: Data Collection and Evaluation Server
- VQA: Visual Question Answering
- T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
- Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
- Simple Baseline for Visual Question Answering
- Show and Tell: A Neural Image Caption Generator
- Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
- Learning a Recurrent Visual Representation for Image Caption Generation
- Deep Visual-Semantic Alignments for Generating Image Descriptions
- Learning Deep Representations of Fine-grained Visual Descriptions
- From Captions to Visual Concepts and Back
- Person Re-identification Meets Image Search
- Segmentation from Natural Language Expressions
- DualNet: Domain-Invariant Network for Visual Question Answering
Cited by in corpus (21)
- Improving Person Re-identification by Attribute and Identity Learning
- Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification
- RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search
- Dialog-based Interactive Image Retrieval
- HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis
- Text-Based Person Search with Limited Data
- Identity-Aware Textual-Visual Matching with Latent Co-attention
- Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
- Scene Graph Generation from Objects, Phrases and Region Captions
- Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry and Fusion
- Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
- Unsupervised Person Re-Identification: A Systematic Survey of Challenges and Solutions
- Improving One-stage Visual Grounding by Recursive Sub-query Construction
- Cross-domain Human Parsing via Adversarial Feature and Label Adaptation
- Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
- Adversarial Attribute-Image Person Re-identification
- Real-time Visual Object Tracking with Natural Language Description
- EgoReID Dataset: Person Re-identification in Videos Acquired by Mobile Devices with First-Person Point-of-View
- Natural Language Person Search Using Deep Reinforcement Learning
- Person Search in Videos with One Portrait Through Visual and Temporal Links
- MGD-GAN: Text-to-Pedestrian generation through Multi-Grained Discrimination