Neural Image Compression and Explanation
arXiv:1908.08988
Abstract
Explaining the prediction of deep neural networks (DNNs) and semantic image compression are two active research areas of deep learning with a numerous of applications in decision-critical systems, such as surveillance cameras, drones and self-driving cars, where interpretable decision is critical and storage/network bandwidth is limited. In this paper, we propose a novel end-to-end Neural Image Compression and Explanation (NICE) framework that learns to (1) explain the predictions of convolutional neural networks (CNNs), and (2) subsequently compress the input images for efficient storage or transmission. Specifically, NICE generates a sparse mask over an input image by attaching a stochastic binary gate to each pixel of the image, whose parameters are learned through the interaction with the CNN classifier to be explained. The generated mask is able to capture the saliency of each pixel measured by its influence to the final prediction of CNN; it can also be used to produce a mixed-resolution image, where important pixels maintain their original high resolution and insignificant background pixels are subsampled to a low resolution. The produced images achieve a high compression rate (e.g., about 0.6x of original image file size), while retaining a similar classification accuracy. Extensive experiments across multiple image classification benchmarks demonstrate the superior performance of NICE compared to the state-of-the-art methods in terms of explanation quality and semantic image compression rate. Our code is available at: https://github.com/lxuniverse/NICE.
Published as a journal paper at IEEE Access 2020
References in corpus (9)
- Learning Important Features Through Propagating Activation Differences
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Understanding Neural Networks through Representation Erasure
- Lossy Image Compression with Compressive Autoencoders
- REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
- Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
- Exploring Neural Transducers for End-to-End Speech Recognition
- Stochastic Variational Optimization
- Interpreting Deep Neural Networks Through Variable Importance