Learning from Incomplete Features by Simultaneous Training of Neural Networks and Sparse Coding

Abstract

In this paper, the problem of training a classifier on a dataset with incomplete features is addressed. We assume that different subsets of features (random or structured) are available at each data instance. This situation typically occurs in the applications when not all the features are collected for every data sample. A new supervised learning method is developed to train a general classifier, such as a logistic regression or a deep neural network, using only a subset of features per sample, while assuming sparse representations of data vectors on an unknown dictionary. Sufficient conditions are identified, such that, if it is possible to train a classifier on incomplete observations so that their reconstructions are well separated by a hyperplane, then the same classifier also correctly separates the original (unobserved) data samples. Extensive simulation results on synthetic and well-known datasets are presented that validate our theoretical findings and demonstrate the effectiveness of the proposed method compared to traditional data imputation approaches and one state-of-the-art algorithm.

arXiv comment11 pages, 7 figures, paper accepted for presentation at L2ID Workshop at CVPR 2021 (19-25 June, 2021)

References in corpus (8)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Supervised Dictionary Learning
• Processing of missing data by neural networks
• On the consistency of supervised learning with missing values
• Efficient EM Training of Gaussian Mixtures with Missing Data
• Classification with Low Rank and Missing Data
• Active Feature Acquisition with Supervised Matrix Completion