3D PersonVLAD: Learning Deep Global Representations for Video-based Person Re-identification

Abstract

In this paper, we introduce a global video representation to video-based person re-identification (re-ID) that aggregates local 3D features across the entire video extent. Most of the existing methods rely on 2D convolutional networks (ConvNets) to extract frame-wise deep features which are pooled temporally to generate the video-level representations. However, 2D ConvNets lose temporal input information immediately after the convolution, and a separate temporal pooling is limited in capturing human motion in shorter sequences. To this end, we present a \textit{global} video representation (3D PersonVLAD), complementary to 3D ConvNets as a novel layer to capture the appearance and motion dynamics in full-length videos. However, encoding each video frame in its entirety and computing an aggregate global representation across all frames is tremendously challenging due to occlusions and misalignments. To resolve this, our proposed network is further augmented with 3D part alignment module to learn local features through soft-attention module. These attended features are statistically aggregated to yield identity-discriminative representations. Our global 3D features are demonstrated to achieve state-of-the-art results on three benchmark datasets: MARS \cite{MARS}, iLIDS-VID \cite{VideoRanking}, and PRID 2011

arXiv commentAccepted to appear at IEEE Transactions on Neural Networks and Learning Systems

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Unsupervised Learning of Video Representations using LSTMs
• Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval
• Deep Adaptive Feature Embedding with Local Sample Distributions for Person Re-identification
• Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval
• Pose Invariant Embedding for Deep Person Re-identification
• What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification
• Deep Recurrent Convolutional Networks for Video-based Person Re-identification: An End-to-End Approach
• Towards Good Practices on Building Effective CNN Baseline Model for Person Re-identification

Cited by in corpus (29)

• Adaptive Graph Representation Learning for Video Person Re-identification
• Few-Shot Deep Adversarial Learning for Video-based Person Re-identification
• Delta Descriptors: Change-Based Place Representation for Robust Visual Localization
• Distance-based Camera Network Topology Inference for Person Re-identification
• Enhancing Visible-Infrared Person Re-identification with Modality- and Instance-aware Visual Prompt Learning
• Attribute-guided Feature Learning Network for Vehicle Re-identification
• Exploring Image Enhancement for Salient Object Detection in Low Light Images
• Kinematics Modeling Network for Video-based Human Pose Estimation
• SFANet: A Spectrum-aware Feature Augmentation Network for Visible-Infrared Person Re-Identification
• AirObject: A Temporally Evolving Graph Embedding for Object Identification
• Cross-Entropy Adversarial View Adaptation for Person Re-identification
• Discriminative Feature and Dictionary Learning with Part-aware Model for Vehicle Re-identification
• Vehicle Re-ID for Surround-view Camera System
• Eliminating cross-camera bias for vehicle re-identification
• Person Re-Identification using Deep Learning Networks: A Systematic Review
• Cross Domain Knowledge Transfer for Unsupervised Vehicle Re-identification
• Purifying Real Images with an Attention-guided Style Transfer Network for Gaze Estimation
• An Item Recommendation Approach by Fusing Images based on Neural Networks
• PAC-GAN: An Effective Pose Augmentation Scheme for Unsupervised Cross-View Person Re-identification
• Deep Visual Waterline Detection within Inland Marine Environment
• A fast online cascaded regression algorithm for face alignment
• Non-rigid 3D shape retrieval based on multi-view metric learning
• Self-Weighted Multiview Metric Learning by Maximizing the Cross Correlations
• Anomaly detecting and ranking of the cloud computing platform by multi-view learning
• Multi-feature Distance Metric Learning for Non-rigid 3D Shape Retrieval
• Auto-weighted Mutli-view Sparse Reconstructive Embedding
• A Targeted Acceleration and Compression Framework for Low bit Neural Networks
• Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization
• Fast Pedestrian Detection based on T-CENTRIST in infrared image