Sampling Equivariant Self-attention Networks for Object Detection in Aerial Images
arXiv:2111.03420
Abstract
Objects in aerial images have greater variations in scale and orientation than in typical images, so detection is more difficult. Convolutional neural networks use a variety of frequency- and orientation-specific kernels to identify objects subject to different transformations; these require many parameters. Sampling equivariant networks can adjust sampling from input feature maps according to the transformation of the object, allowing a kernel to extract features of an object under different transformations. Doing so requires fewer parameters, and makes the network more suitable for representing deformable objects, like those in aerial images. However, methods like deformable convolutional networks can only provide sampling equivariance under certain circumstances, because of the locations used for sampling. We propose sampling equivariant self-attention networks which consider self-attention restricted to a local image patch as convolution sampling with masks instead of locations, and design a transformation embedding module to further improve the equivariant sampling ability. We also use a novel randomized normalization module to tackle overfitting due to limited aerial image data. We show that our model (i) provides significantly better sampling equivariance than existing methods, without additional supervision, (ii) provides improved classification on ImageNet, and (iii) achieves state-of-the-art results on the DOTA dataset, without increased computation.
References in corpus (24)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- Gliding vertex on the horizontal bounding box for multi-oriented object detection
- MMDetection: Open MMLab Detection Toolbox and Benchmark
- Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges
- Automatic Ship Detection of Remote Sensing Images from Google Earth in Complex Scenes Based on Multi-Scale Rotation Dense Feature Pyramid Networks
- Group Equivariant Convolutional Networks
- Stand-Alone Self-Attention in Vision Models
- Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss
- Learning a Rotation Invariant Detector with Rotatable Bounding Box
- R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object
- ConvBERT: Improving BERT with Span-based Dynamic Convolution
- End-to-End Object Detection with Adaptive Clustering Transformer
- Locally Scale-Invariant Convolutional Neural Networks
- Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
- End-to-End Video Instance Segmentation with Transformers
- Learning Modulated Loss for Rotated Object Detection
- Spatial Transformer Networks
- Is Attention Better Than Matrix Decomposition?
- Are Sixteen Heads Really Better than One?
- End-to-End Human Pose and Mesh Reconstruction with Transformers
- Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation
- FGSD: A Dataset for Fine-Grained Ship Detection in High Resolution Satellite Images
- Dense Label Encoding for Boundary Discontinuity Free Rotation Detection