Improving Inference for Neural Image Compression
arXiv:2006.04240
Abstract
We consider the problem of lossy image compression with deep latent variable models. State-of-the-art methods build on hierarchical variational autoencoders (VAEs) and learn inference networks to predict a compressible latent representation of each data point. Drawing on the variational inference perspective on compression, we identify three approximation gaps which limit performance in the conventional approach: an amortization gap, a discretization gap, and a marginalization gap. We propose remedies for each of these three limitations based on ideas related to iterative inference, stochastic annealing for discrete optimization, and bits-back coding, resulting in the first application of bits-back coding to lossy compression. In our experiments, which include extensive baseline comparisons and ablation studies, we achieve new state-of-the-art performance on lossy image compression using an established VAE architecture, by changing only the inference method.
arXiv comment9 pages + detailed supplement with additional results; various typos corrected. Camera-ready version paper at NeurIPS 2020
References in corpus (17)
- Categorical Reparameterization with Gumbel-Softmax
- Neural Discrete Representation Learning
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Variational image compression with a scale hyperprior
- End-to-end Optimized Image Compression
- The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
- Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
- Joint Autoregressive and Hierarchical Priors for Learned Image Compression
- Context-adaptive Entropy Model for End-to-end Optimized Image Compression
- Video Compression With Rate-Distortion Autoencoders
- Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
- Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables
- Practical Lossless Compression with Latent Variables using Bits Back Coding
- Compression with Flows via Local Bits-Back Coding
- Computationally Efficient Neural Image Compression
- HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models
- Hierarchical Autoregressive Modeling for Neural Video Compression
Cited by in corpus (12)
- Supervised Compression for Resource-Constrained Edge Computing Systems
- Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
- Overfitting for Fun and Profit: Instance-Adaptive Data Compression
- Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression
- Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set
- Soft then Hard: Rethinking the Quantization in Neural Image Compression
- Towards Empirical Sandwich Bounds on the Rate-Distortion Function
- Hierarchical Autoregressive Modeling for Neural Video Compression
- DeepMapping: Learned Data Mapping for Lossless Compression and Efficient Lookup
- Learning End-to-End Lossy Image Compression: A Benchmark
- Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
- Variational Bayesian Quantization