Towards a Theoretical Understanding of the Robustness of Variational Autoencoders

Abstract

We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. While previous work has developed algorithmic approaches to attacking and defending VAEs, there remains a lack of formalization for what it means for a VAE to be robust. To address this, we develop a novel criterion for robustness in probabilistic models: $r$-robustness. We then use this to construct the first theoretical results for the robustness of VAEs, deriving margins in the input space for which we can provide guarantees about the resulting reconstruction. Informally, we are able to define a region within which any perturbation will produce a reconstruction that is similar to the original reconstruction. To support our analysis, we show that VAEs trained using disentangling methods not only score well under our robustness metrics, but that the reasons for this can be interpreted through our theoretical results.

arXiv comment8 pages

References in corpus (14)

• Auto-Encoding Variational Bayes
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Understanding disentangling in $\beta$-VAE
• Robust Large Margin Deep Neural Networks
• Towards the first adversarially robust neural network model on MNIST
• DARLA: Improving Zero-Shot Transfer in Reinforcement Learning
• A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance
• Adversarial Images for Variational Autoencoders
• Assigning a value to a power likelihood in a general Bayesian model
• Adversarial Attacks on Variational Autoencoders
• Adversarial Attacks, Regression, and Numerical Stability Regularization
• On Implicit Regularization in $\beta$-VAEs
• Improving VAEs' Robustness to Adversarial Attack