iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression
arXiv:2103.16211
Abstract
It is nontrivial to store rapidly growing big data nowadays, which demands high-performance lossless compression techniques. Likelihood-based generative models have witnessed their success on lossless compression, where flow based models are desirable in allowing exact data likelihood optimisation with bijective mappings. However, common continuous flows are in contradiction with the discreteness of coding schemes, which requires either 1) imposing strict constraints on flow models that degrades the performance or 2) coding numerous bijective mapping errors which reduces the efficiency. In this paper, we investigate volume preserving flows for lossless compression and show that a bijective mapping without error is possible. We propose Numerical Invertible Volume Preserving Flow (iVPF) which is derived from the general volume preserving flows. By introducing novel computation algorithms on flow models, an exact bijective mapping is achieved without any numerical error. We also propose a lossless compression algorithm based on iVPF. Experiments on various datasets show that the algorithm based on iVPF achieves state-of-the-art compression ratio over lightweight compression algorithms.
arXiv commentAccepted for CVPR 2021
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Conditional Image Generation with PixelCNN Decoders
- NICE: Non-linear Independent Components Estimation
- Density estimation using Real NVP
- Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding
- PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications
- Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables
- Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
- Practical Lossless Compression with Latent Variables using Bits Back Coding
- IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
- Compression with Flows via Local Bits-Back Coding
- Lossless Image Compression through Super-Resolution
- Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
- Learning Discrete Distributions by Dequantization
- HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Models