TransReID: Transformer-based Object Re-Identification

Abstract

Extracting robust feature representation is one of the key challenges in object re-identification (ReID). Although convolution neural network (CNN)-based methods have achieved great success, they only process one local neighborhood at a time and suffer from information loss on details caused by convolution and downsampling operators (e.g. pooling and strided convolution). To overcome these limitations, we propose a pure transformer-based object ReID framework named TransReID. Specifically, we first encode an image as a sequence of patches and build a transformer-based strong baseline with a few critical improvements, which achieves competitive results on several ReID benchmarks with CNN-based methods. To further enhance the robust feature learning in the context of transformers, two novel modules are carefully designed. (i) The jigsaw patch module (JPM) is proposed to rearrange the patch embeddings via shift and patch shuffle operations which generates robust features with improved discrimination ability and more diversified coverage. (ii) The side information embeddings (SIE) is introduced to mitigate feature bias towards camera/view variations by plugging in learnable embeddings to incorporate these non-visual clues. To the best of our knowledge, this is the first work to adopt a pure transformer for ReID research. Experimental results of TransReID are superior promising, which achieve state-of-the-art performance on both person and vehicle ReID benchmarks.

Code is available at https://github.com/heshuting555/TransReID

References in corpus (13)

• Transformers in Vision: A Survey
• A Discriminatively Learned CNN Embedding for Person Re-identification
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Deep Representation Learning with Part Loss for Person Re-Identification
• GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval
• Reducing Transformer Depth on Demand with Structured Dropout
• Toward Transformer-Based Object Detection
• Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification
• Training with Quantization Noise for Extreme Model Compression
• Second-order Non-local Attention Networks for Person Re-identification
• How to Start Training: The Effect of Initialization and Architecture
• Identity-Guided Human Semantic Parsing for Person Re-Identification
• Looking GLAMORous: Vehicle Re-Id in Heterogeneous Cameras Networks with Global and Local Attention

Cited by in corpus (19)

• Transformers in Vision: A Survey
• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
• CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
• Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
• Multiscale Vision Transformers
• A Video Is Worth Three Views: Trigeminal Transformers for Video-based Person Re-identification
• Transformer-Based Source-Free Domain Adaptation
• MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
• OH-Former: Omni-Relational High-Order Transformer for Person Re-Identification
• Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation
• HAT: Hierarchical Aggregation Transformers for Person Re-identification
• GiT: Graph Interactive Transformer for Vehicle Re-identification
• TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection
• KVT: k-NN Attention for Boosting Vision Transformers
• Towards Discriminative Representation Learning for Unsupervised Person Re-identification
• Salient Object Ranking with Position-Preserved Attention
• City-Scale Multi-Camera Vehicle Tracking Guided by Crossroad Zones
• An Empirical Study of Vehicle Re-Identification on the AI City Challenge