Universally Quantized Neural Compression

Abstract

A popular approach to learning encoders for lossy compression is to use additive uniform noise during training as a differentiable approximation to test-time quantization. We demonstrate that a uniform noise channel can also be implemented at test time using universal quantization (Ziv, 1985). This allows us to eliminate the mismatch between training and test phases while maintaining a completely differentiable loss function. Implementing the uniform noise channel is a special case of the more general problem of communicating a sample, which we prove is computationally hard if we do not make assumptions about its distribution. However, the uniform special case is efficient as well as easy to implement and thus of great interest from a practical point of view. Finally, we show that quantization can be obtained as a limiting case of a soft quantizer applied to the uniform noise channel, bridging compression with and without quantization.

arXiv commentAuthors contributed equally

References in corpus (6)

• Auto-Encoding Variational Bayes
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Variational image compression with a scale hyperprior
• End-to-end Optimized Image Compression
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Joint Autoregressive and Hierarchical Priors for Learned Image Compression

Cited by in corpus (9)

• CompressAI: a PyTorch library and evaluation platform for end-to-end compression research
• Lossy Image Compression with Quantized Hierarchical VAEs
• IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
• Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
• On the advantages of stochastic encoders
• Adaptive Greedy Rejection Sampling
• Learned Block-based Hybrid Image Compression
• DietCNN: Multiplication-free Inference for Quantized CNNs
• Principal Bit Analysis: Autoencoding with Schur-Concave Loss