Parameter-Efficient Person Re-identification in the 3D Space
arXiv:2006.04569 · doi:10.1109/TNNLS.2022.3214834
Abstract
People live in a 3D world. However, existing works on person re-identification (re-id) mostly consider the semantic representation learning in a 2D space, intrinsically limiting the understanding of people. In this work, we address this limitation by exploring the prior knowledge of the 3D body structure. Specifically, we project 2D images to a 3D space and introduce a novel parameter-efficient Omni-scale Graph Network (OG-Net) to learn the pedestrian representation directly from 3D point clouds. OG-Net effectively exploits the local information provided by sparse 3D points and takes advantage of the structure and appearance information in a coherent manner. With the help of 3D geometry information, we can learn a new type of deep re-id feature free from noisy variants, such as scale and viewpoint. To our knowledge, we are among the first attempts to conduct person re-identification in the 3D space. We demonstrate through extensive experiments that the proposed method (1) eases the matching difficulty in the traditional 2D space, (2) exploits the complementary information of 2D appearance and 3D structure, (3) achieves competitive results with limited parameters on four large-scale person re-id datasets, and (4) has good scalability to unseen datasets. Our code, models and generated 3D human data are publicly available at https://github.com/layumi/person-reid-3d .
arXiv commentThe code is available at https://github.com/layumi/person-reid-3d
References in corpus (30)
- Adam: A Method for Stochastic Optimization
- Inductive Representation Learning on Large Graphs
- In Defense of the Triplet Loss for Person Re-Identification
- Self-Attention Generative Adversarial Networks
- Squeeze-and-Excitation Networks
- PCT: Point cloud transformer
- On the Convergence of Adam and Beyond
- Learning Discriminative Features with Multiple Granularities for Person Re-Identification
- Person Re-identification: Past, Present and Future
- Improving Person Re-identification by Attribute and Identity Learning
- A Discriminatively Learned CNN Embedding for Person Re-identification
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- Pedestrian Alignment Network for Large-scale Person Re-identification
- Dual-Path Convolutional Image-Text Embeddings with Instance Loss
- AlignedReID: Surpassing Human-Level Performance in Person Re-Identification
- Large-Margin Softmax Loss for Convolutional Neural Networks
- FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
- Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification
- VehicleNet: Learning Robust Visual Representation for Vehicle Re-identification
- Adaptive Exploration for Unsupervised Person Re-Identification
- Deep Learning for Person Re-identification: A Survey and Outlook
- Multi-pseudo Regularized Label for Generated Data in Person Re-Identification
- RPC: A Large-Scale Retail Product Checkout Dataset
- DeepGCNs: Can GCNs Go as Deep as CNNs?
- Circle Loss: A Unified Perspective of Pair Similarity Optimization
- The Devil is in the Middle: Exploiting Mid-level Representations for Cross-Domain Instance Matching
- Cross-Domain Complementary Learning Using Pose for Multi-Person Part Segmentation
- Learning Disentangled Representation for Robust Person Re-identification
- Simulating Content Consistent Vehicle Datasets with Attribute Descent
- MagnifierNet: Towards Semantic Adversary and Fusion for Person Re-identification