What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification

Abstract

Matching pedestrians across disjoint camera views, known as person re-identification (re-id), is a challenging problem that is of importance to visual recognition and surveillance. Most existing methods exploit local regions within spatial manipulation to perform matching in local correspondence. However, they essentially extract \emph{fixed} representations from pre-divided regions for each image and perform matching based on the extracted representation subsequently. For models in this pipeline, local finer patterns that are crucial to distinguish positive pairs from negative ones cannot be captured, and thus making them underperformed. In this paper, we propose a novel deep multiplicative integration gating function, which answers the question of \emph{what-and-where to match} for effective person re-id. To address \emph{what} to match, our deep network emphasizes common local patterns by learning joint representations in a multiplicative way. The network comprises two Convolutional Neural Networks (CNNs) to extract convolutional activations, and generates relevant descriptors for pedestrian matching. This thus, leads to flexible representations for pair-wise images. To address \emph{where} to match, we combat the spatial misalignment by performing spatially recurrent pooling via a four-directional recurrent neural network to impose spatial dependency over all positions with respect to the entire image. The proposed network is designed to be end-to-end trainable to characterize local pairwise feature interactions in a spatially aligned manner. To demonstrate the superiority of our method, extensive experiments are conducted over three benchmark data sets: VIPeR, CUHK03 and Market-1501.

arXiv commentPublished at Pattern Recognition, Elsevier

References in corpus (24)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• ParseNet: Looking Wider to See Better
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
• End-to-End Comparative Attention Networks for Person Re-identification
• Diversified Visual Attention Networks for Fine-Grained Object Classification
• Person Re-Identification by Camera Correlation Aware Feature Augmentation
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks
• PersonNet: Person Re-identification with Deep Convolutional Neural Networks
• Deep Adaptive Feature Embedding with Local Sample Distributions for Person Re-identification
• Pose Invariant Embedding for Deep Person Re-identification
• Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification
• Learning a Discriminative Null Space for Person Re-identification
• Structured Deep Hashing with Convolutional Neural Networks for Fast Person Re-identification
• Gated Siamese Convolutional Neural Network Architecture for Human Re-Identification
• Embedding Deep Metric for Person Re-identication A Study Against Large Variations
• A Siamese Long Short-Term Memory Architecture for Human Re-Identification
• On Multiplicative Integration with Recurrent Neural Networks
• Scalable Person Re-identification on Supervised Smoothed Manifold
• Nonlinear Local Metric Learning for Person Re-identification
• Person Re-identification with Correspondence Structure Learning

Cited by in corpus (48)

• Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization
• 3D PersonVLAD: Learning Deep Global Representations for Video-based Person Re-identification
• Few-Shot Deep Adversarial Learning for Video-based Person Re-identification
• Structured Deep Hashing with Convolutional Neural Networks for Fast Person Re-identification
• Multi-Domain Adversarial Feature Generalization for Person Re-Identification
• Deep-Person: Learning Discriminative Deep Features for Person Re-Identification
• Hierarchical Information Quadtree: Efficient Spatial Temporal Image Search for Multimedia Stream
• Distance-based Camera Network Topology Inference for Person Re-identification
• Attribute-guided Feature Learning Network for Vehicle Re-identification
• Deep Co-attention based Comparators For Relative Representation Learning in Person Re-identification
• Where to Focus: Deep Attention-based Spatially Recurrent Bilinear Networks for Fine-Grained Visual Recognition
• Cross-Entropy Adversarial View Adaptation for Person Re-identification
• Where-and-When to Look: Deep Siamese Attention Networks for Video-based Person Re-identification
• Eliminating cross-camera bias for vehicle re-identification
• Cross Domain Knowledge Learning with Dual-branch Adversarial Network for Vehicle Re-identification
• Devil's in the Details: Aligning Visual Clues for Conditional Embedding in Person Re-Identification
• Hierarchical One Permutation Hashing: Efficient Multimedia Near Duplicate Detection
• STADB: A Self-Thresholding Attention Guided ADB Network for Person Re-identification
• Saliency guided deep network for weakly-supervised image segmentation
• Semantically Selective Augmentation for Deep Compact Person Re-Identification
• PAC-GAN: An Effective Pose Augmentation Scheme for Unsupervised Cross-View Person Re-identification
• Deep neural network-based classification model for Sentiment Analysis
• Self-Attention Recurrent Network for Saliency Detection
• Person Re-Identification using Deep Learning Networks: A Systematic Review
• An Efficient Approach for Geo-Multimedia Cross-Modal Retrieval
• Guiding Intelligent Surveillance System by learning-by-synthesis gaze estimation
• Efficient Region of Visual Interests Search for Geo-multimedia Data
• Efficient Interactive Search for Geo-tagged Multimedia Data
• A hybrid index model for efficient spatio-temporal search in HBase
• A Targeted Acceleration and Compression Framework for Low bit Neural Networks
• Using Context Information to Enhance Simple Question Answering
• Anomaly detecting and ranking of the cloud computing platform by multi-view learning
• Multi-feature Distance Metric Learning for Non-rigid 3D Shape Retrieval
• Auto-weighted Mutli-view Sparse Reconstructive Embedding
• Structured Mean-field Variational Inference and Learning in Winner-take-all Spiking Neural Networks
• Hierarchical Attention Network for Action Segmentation
• HOC-Tree: A Novel Index for efficient Spatio-temporal Range Search
• ConCAD: Contrastive Learning-based Cross Attention for Sleep Apnea Detection
• Fast Pedestrian Detection based on T-CENTRIST in infrared image
• GPU based Parallel Optimization for Real Time Panoramic Video Stitching
• CNN-VWII: An Efficient Approach for Large-Scale Video Retrieval by Image Queries
• The combination of context information to enhance simple question answering
• Efficient Multimedia Similarity Measurement Using Similar Elements
• Efficient Continuous Top-$k$ Geo-Image Search on Road Network
• Robust Tracking via Weighted Online Extreme Learning Machine
• A Filter of Minhash for Image Similarity Measures
• Temporal Activity Path Based Character Correction in Social Networks
• Efficient Top K Temporal Spatial Keyword Search