Towards Robust Active Feature Acquisition

Abstract

Truly intelligent systems are expected to make critical decisions with incomplete and uncertain data. Active feature acquisition (AFA), where features are sequentially acquired to improve the prediction, is a step towards this goal. However, current AFA models all deal with a small set of candidate features and have difficulty scaling to a large feature space. Moreover, they are ignorant about the valid domains where they can predict confidently, thus they can be vulnerable to out-of-distribution (OOD) inputs. In order to remedy these deficiencies and bring AFA models closer to practical use, we propose several techniques to advance the current AFA approaches. Our framework can easily handle a large number of features using a hierarchical acquisition policy and is more robust to OOD inputs with the help of an OOD detector for partially observed data. Extensive experiments demonstrate the efficacy of our framework over strong baselines.

References in corpus (14)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Generative Modeling by Estimating Gradients of the Data Distribution
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Deep Anomaly Detection with Outlier Exposure
• Masked Autoregressive Flow for Density Estimation
• WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness
• Verified Uncertainty Calibration
• On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty
• Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
• EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE
• Active Feature Acquisition with Generative Surrogate Models
• Active Information Acquisition