Towards Visually Explaining Variational Autoencoders
arXiv:1911.07389
Abstract
Recent advances in Convolutional Neural Network (CNN) model interpretability have led to impressive progress in visualizing and understanding model predictions. In particular, gradient-based visual attention methods have driven much recent effort in using visual attention maps as a means for visual explanations. A key problem, however, is these methods are designed for classification and categorization tasks, and their extension to explaining generative models, e.g. variational autoencoders (VAE) is not trivial. In this work, we take a step towards bridging this crucial gap, proposing the first technique to visually explain VAEs by means of gradient-based attention. We present methods to generate visual attention from the learned latent space, and also demonstrate such attention explanations serve more than just explaining VAE predictions. We show how these attention maps can be used to localize anomalies in images, demonstrating state-of-the-art performance on the MVTec-AD dataset. We also show how they can be infused into model training, helping bootstrap the VAE into learning improved latent space disentanglement, demonstrated on the Dsprites dataset.
arXiv comment10 pages, 9 figures, 2 tables, CVPR 2020
References in corpus (12)
- Auto-Encoding Variational Bayes
- Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
- Self-Attention Generative Adversarial Networks
- Deep Learning for Anomaly Detection: A Survey
- Pose Guided Person Image Generation
- Disentangling by Factorising
- InfoVAE: Information Maximizing Variational Autoencoders
- Unsupervised Attention-guided Image to Image Translation
- Anomaly Detection using One-Class Neural Networks
- Learning Generative Models with Visual Attention
- Disentangling Factors of Variation via Generative Entangling
- Accurate Esophageal Gross Tumor Volume Segmentation in PET/CT using Two-Stream Chained 3D Deep Network Fusion