Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach

Abstract

Multi-view learning (MVL) leverages multiple sources or views of data to enhance machine learning model performance and robustness. This approach has been successfully used in the Earth Observation (EO) domain, where views have a heterogeneous nature and can be affected by missing data. Despite the negative effect that missing data has on model predictions, the ML literature has used it as an augmentation technique to improve model generalization, like masking the input data. Inspired by this, we introduce novel methods for EO applications tailored to MVL with missing views. Our methods integrate the combination of a set to simulate all combinations of missing views as different training samples. Instead of replacing missing data with a numerical value, we use dynamic merge functions, like average, and more complex ones like Transformer. This allows the MVL model to entirely ignore the missing views, enhancing its predictive robustness. We experiment on four EO datasets with temporal and static views, including state-of-the-art methods from the EO domain. The results indicate that our methods improve model robustness under conditions of moderate missingness, and improve the predictive performance when all views are present. The proposed methods offer a single adaptive solution to operate effectively with any combination of available views.

References in corpus (12)

• More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• SAITS: Self-Attention-based Imputation for Time Series
• BRITS: Bidirectional Recurrent Imputation for Time Series
• Compute and Energy Consumption Trends in Deep Learning Inference
• EmbraceNet: A robust deep learning architecture for multimodal classification
• SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
• Dropout as data augmentation
• Adaptive Fusion of Multi-view Remote Sensing data for Optimal Sub-field Crop Yield Prediction
• Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications
• Lightweight, Pre-trained Transformers for Remote Sensing Timeseries
• Mission Critical -- Satellite Data is a Distinct Modality in Machine Learning
• Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

Cited by in corpus (1)

• Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration