MAIN: Multihead-Attention Imputation Networks

Abstract

The problem of missing data, usually absent incurated and competition-standard datasets, is an unfortunate reality for most machine learning models used in industry applications. Recent work has focused on understanding the nature and the negative effects of such phenomena, while devising solutions for optimal imputation of the missing data, using both discriminative and generative approaches. We propose a novel mechanism based on multi-head attention which can be applied effortlessly in any model and achieves better downstream performance without the introduction of the full dataset in any part of the modeling pipeline. Our method inductively models patterns of missingness in the input data in order to increase the performance of the downstream task. Finally, after evaluating our method against baselines for a number of datasets, we found performance gains that tend to be larger in scenarios of high missingness.

8 pages, 7 figures

Cited by in corpus (4)

• LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
• Computational Solar Energy -- Ensemble Learning Methods for Prediction of Solar Power Generation based on Meteorological Parameters in Eastern India
• Deep Integrated Pipeline of Segmentation Guided Classification of Breast Cancer from Ultrasound Images
• Markerless human pose estimation for biomedical applications: a survey