Compression with Flows via Local Bits-Back Coding
arXiv:1905.08500
Abstract
Likelihood-based generative models are the backbones of lossless compression due to the guaranteed existence of codes with lengths close to negative log likelihood. However, there is no guaranteed existence of computationally efficient codes that achieve these lengths, and coding algorithms must be hand-tailored to specific types of generative models to ensure computational efficiency. Such coding algorithms are known for autoregressive models and variational autoencoders, but not for general types of flow models. To fill in this gap, we introduce local bits-back coding, a new compression technique for flow models. We present efficient algorithms that instantiate our technique for many popular types of flows, and we demonstrate that our algorithms closely achieve theoretical codelengths for state-of-the-art flow models on high-dimensional data.
Published in NeurIPS 2019
Cited by in corpus (16)
- Variational Diffusion Models
- Improving Inference for Neural Image Compression
- Maximum Likelihood Training of Score-Based Diffusion Models
- IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
- Lossless Image Compression through Super-Resolution
- HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models
- Towards Empirical Sandwich Bounds on the Rate-Distortion Function
- Variational Inference with Continuously-Indexed Normalizing Flows
- Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks
- OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression
- Partition and Code: learning how to compress graphs
- Autoregressive Diffusion Models
- Lossless Compression with Latent Variable Models
- Lossless Compression with Probabilistic Circuits
- iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression
- Lossless Image Compression Using a Multi-Scale Progressive Statistical Model