not-MIWAE: Deep Generative Modelling with Missing not at Random Data

Abstract

When a missing process depends on the missing values themselves, it needs to be explicitly modelled and taken into account while doing likelihood-based inference. We present an approach for building and fitting deep latent variable models (DLVMs) in cases where the missing process is dependent on the missing data. Specifically, a deep neural network enables us to flexibly model the conditional distribution of the missingness pattern given the data. This allows for incorporating prior information about the type of missingness (e.g. self-censoring) into the model. Our inference technique, based on importance-weighted variational inference, involves maximising a lower bound of the joint likelihood. Stochastic gradients of the bound are obtained by using the reparameterisation trick both in latent space and data space. We show on various kinds of data sets and missingness patterns that explicitly modelling the missing process can be invaluable.

arXiv commentCamera-ready version for ICLR 2021

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• MissForest - nonparametric missing value imputation for mixed-type data
• Categorical Reparameterization with Gumbel-Softmax
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• GAIN: Missing Data Imputation using Generative Adversarial Nets
• TensorFlow Distributions
• Importance Weighted Autoencoders
• What Is Meant by "Missing at Random"?
• Collaborative Filtering and the Missing at Random Assumption
• Recommendations as Treatments: Debiasing Learning and Evaluation
• Implicit Reparameterization Gradients
• Variational Autoencoder with Arbitrary Conditioning
• Probabilistic symmetries and invariant neural networks
• Importance Weighting and Variational Inference
• EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE
• Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
• Missing Not at Random in Matrix Completion: The Effectiveness of Estimating Missingness Probabilities Under a Low Nuclear Norm Assumption
• Estimation and imputation in Probabilistic Principal Component Analysis with Missing Not At Random data
• Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation
• VAEs in the Presence of Missing Data
• Deep Generative Pattern-Set Mixture Models for Nonignorable Missingness

Cited by in corpus (8)

• Spatiotemporal Tensor Completion for Improved Urban Traffic Imputation
• Deep Generative Imputation Model for Missing Not At Random Data
• Identifiable Generative Models for Missing Not at Random Data Imputation
• A Brief Overview of Unsupervised Neural Speech Representation Learning
• Interpretable Models Capable of Handling Systematic Missingness in Imbalanced Classes and Heterogeneous Datasets
• Model-agnostic out-of-distribution detection using combined statistical tests
• On the Consistency of a Random Forest Algorithm in the Presence of Missing Entries
• EMFlow: Data Imputation in Latent Space via EM and Deep Flow Models