SpecDETR: A transformer-based hyperspectral point object detection network

Abstract

Hyperspectral target detection (HTD) aims to identify specific materials based on spectral information in hyperspectral imagery and can detect extremely small-sized objects, some of which occupy a smaller than one-pixel area. However, existing HTD methods are developed based on per-pixel binary classification, neglecting the three-dimensional cube structure of hyperspectral images (HSIs) that integrates both spatial and spectral dimensions. The synergistic existence of spatial and spectral features in HSIs enable objects to simultaneously exhibit both, yet the per-pixel HTD framework limits the joint expression of these features. In this paper, we rethink HTD from the perspective of spatial-spectral synergistic representation and propose hyperspectral point object detection as an innovative task framework. We introduce SpecDETR, the first specialized network for hyperspectral multi-class point object detection, which eliminates dependence on pre-trained backbone networks commonly required by vision-based object detectors. SpecDETR uses a multi-layer Transformer encoder with self-excited subpixel-scale attention modules to directly extract deep spatial-spectral joint features from hyperspectral cubes. We develop a simulated hyperspectral point object detection benchmark termed SPOD, and for the first time, evaluate and compare the performance of visual object detection networks and HTD methods on hyperspectral point object detection. Extensive experiments demonstrate that our proposed SpecDETR outperforms SOTA visual object detection networks and HTD methods. Our code and dataset are available at https://github.com/ZhaoxuLi123/SpecDETR.

References in corpus (9)

• EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• YOLOv10: Real-Time End-to-End Object Detection
• DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
• Towards Large-Scale Small Object Detection: Survey and Benchmarks
• DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
• You Only Train Once: Learning a General Anomaly Enhancement Network with Random Masks for Hyperspectral Anomaly Detection
• Highly Efficient and Unsupervised Framework for Moving Object Detection in Satellite Videos
• Plant species richness prediction from DESIS hyperspectral data: A comparison study on feature extraction procedures and regression models