Robust Variational Autoencoder

Abstract

Machine learning methods often need a large amount of labeled training data. Since the training data is assumed to be the ground truth, outliers can severely degrade learned representations and performance of trained models. Here we apply concepts from robust statistics to derive a novel variational autoencoder that is robust to outliers in the training data. Variational autoencoders (VAEs) extract a lower-dimensional encoded feature representation from which we can generate new data samples. Robustness of autoencoders to outliers is critical for generating a reliable representation of particular data types in the encoded space when using corrupted training data. Our robust VAE is based on beta-divergence rather than the standard Kullback-Leibler (KL) divergence. Our proposed lower bound lead to a RVAE model that has the same computational complexity as the VAE and contains a single tuning parameter to control the degree of robustness. We demonstrate the performance of our $\beta$-divergence based autoencoder for a range of image datasets, showing improved robustness to outliers both qualitatively and quantitatively. We also illustrate the use of our robust VAE for outlier detection.

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Tutorial on Variational Autoencoders
• Autoencoding beyond pixels using a learned similarity metric
• EMNIST: an extension of MNIST to handwritten letters
• Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Automated Variational Inference in Probabilistic Programming
• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
• The continuous Bernoulli: fixing a pervasive error in variational autoencoders
• Variational Inference based on Robust Divergences
• Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder

Cited by in corpus (5)

• Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder
• Robust Variational Autoencoder for Tabular Data with Beta Divergence
• Novelty Detection via Robust Variational Autoencoding
• Probabilistic Outlier Detection and Generation
• Uncertainty-Aware Learning for Improvements in Image Quality of the Canada-France-Hawaii Telescope