Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification

Abstract

We present AiTLAS: Benchmark Arena -- an open-source benchmark suite for evaluating state-of-the-art deep learning approaches for image classification in Earth Observation (EO). To this end, we present a comprehensive comparative analysis of more than 500 models derived from ten different state-of-the-art architectures and compare them to a variety of multi-class and multi-label classification tasks from 22 datasets with different sizes and properties. In addition to models trained entirely on these datasets, we benchmark models trained in the context of transfer learning, leveraging pre-trained model variants, as it is typically performed in practice. All presented approaches are general and can be easily extended to many other remote sensing image classification tasks not considered in this study. To ensure reproducibility and facilitate better usability and further developments, all of the experimental resources including the trained models, model configurations, and processing details of the datasets (with their corresponding splits used for training and evaluating the models) are publicly available on the repository: https://github.com/biasvariancelabs/aitlas-arena

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• How transferable are features in deep neural networks?
• Deep learning in remote sensing: a review
• Remote Sensing Image Scene Classification: Benchmark and State of the Art
• A Survey of the Recent Architectures of Deep Convolutional Neural Networks
• AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification
• MLP-Mixer: An all-MLP Architecture for Vision
• Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities
• A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community
• PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval
• BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding
• Building Instance Classification Using Street View Images
• Land Cover Classification via Multi-temporal Spatial Data by Recurrent Neural Networks
• An Empirical Study of Remote Sensing Pretraining
• BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval
• Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification
• A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
• Relation Network for Multi-label Aerial Image Classification

Cited by in corpus (2)

• In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
• Adaptive Gradient Calibration for Single-Positive Multi-Label Learning in Remote Sensing Image Scene Classification