A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization
arXiv:2201.09206 · doi:10.1109/TCSVT.2021.3135013
Abstract
Cross-view geo-localization is a task of matching the same geographic image from different views, e.g., unmanned aerial vehicle (UAV) and satellite. The most difficult challenges are the position shift and the uncertainty of distance and scale. Existing methods are mainly aimed at digging for more comprehensive fine-grained information. However, it underestimates the importance of extracting robust feature representation and the impact of feature alignment. The CNN-based methods have achieved great success in cross-view geo-localization. However it still has some limitations, e.g., it can only extract part of the information in the neighborhood and some scale reduction operations will make some fine-grained information lost. In particular, we introduce a simple and efficient transformer-based structure called Feature Segmentation and Region Alignment (FSRA) to enhance the model's ability to understand contextual information as well as to understand the distribution of instances. Without using additional supervisory information, FSRA divides regions based on the heat distribution of the transformer's feature map, and then aligns multiple specific regions in different views one on one. Finally, FSRA integrates each region into a set of feature representations. The difference is that FSRA does not divide regions manually, but automatically based on the heat distribution of the feature map. So that specific instances can still be divided and aligned when there are significant shifts and scale changes in the image. In addition, a multiple sampling strategy is proposed to overcome the disparity in the number of satellite images and that of images from other sources. Experiments show that the proposed method has superior performance and achieves the state-of-the-art in both tasks of drone view target localization and drone navigation. Code will be released at https://github.com/Dmmm1997/FSRA
arXiv comment14 pages, 13 figures, IEEE Transactions on Circuits and Systems for Video Technology
References in corpus (15)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- A Survey on Visual Transformer
- Learning Discriminative Features with Multiple Granularities for Person Re-Identification
- A Discriminatively Learned CNN Embedding for Person Re-identification
- Pedestrian Alignment Network for Large-scale Person Re-identification
- Dual-Path Convolutional Image-Text Embeddings with Instance Loss
- End-to-End Comparative Attention Networks for Person Re-identification
- Deep Representation Learning with Part Loss for Person Re-Identification
- Vision Meets Drones: A Challenge
- Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization
- TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
- Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification
- Multi-view analysis of unregistered medical images using cross-view transformers
- ConvTransformer: A Convolutional Transformer Network for Video Frame Synthesis
- Cross-view Geo-localization with Evolving Transformer