Where-and-When to Look: Deep Siamese Attention Networks for Video-based Person Re-identification
arXiv:1808.01911
Abstract
Video-based person re-identification (re-id) is a central application in surveillance systems with significant concern in security. Matching persons across disjoint camera views in their video fragments is inherently challenging due to the large visual variations and uncontrolled frame rates. There are two steps crucial to person re-id, namely discriminative feature learning and metric learning. However, existing approaches consider the two steps independently, and they do not make full use of the temporal and spatial information in videos. In this paper, we propose a Siamese attention architecture that jointly learns spatiotemporal video representations and their similarity metrics. The network extracts local convolutional features from regions of each frame, and enhance their discriminative capability by focusing on distinct regions when measuring the similarity with another pedestrian video. The attention mechanism is embedded into spatial gated recurrent units to selectively propagate relevant features and memorize their spatial dependencies through the network. The model essentially learns which parts (\emph{where}) from which frames (\emph{when}) are relevant and distinctive for matching persons and attaches higher importance therein. The proposed Siamese model is end-to-end trainable to jointly learn comparable hidden representations for paired pedestrian videos and their similarity value. Extensive experiments on three benchmark datasets show the effectiveness of each component of the proposed deep network while outperforming state-of-the-art methods.
arXiv commentAppearing in IEEE Transactions on Multimedia. arXiv admin note: text overlap with arXiv:1606.01609
References in corpus (10)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization
- Action Recognition using Visual Attention
- Deep Transfer Learning for Person Re-identification
- Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval
- What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification
- SVDNet for Pedestrian Retrieval
- Iterative Views Agreement: An Iterative Low-Rank based Structured Optimization Method to Multi-View Spectral Clustering
- Recurrent Network Models for Human Dynamics