Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff

Abstract

Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is not a synonym for "high perceptual quality", and in fact optimization of one often comes at the expense of the other. In light of this understanding, it is natural to seek for a generalization of rate-distortion theory which takes perceptual quality into account. In this paper, we adopt the mathematical definition of perceptual quality recently proposed by Blau & Michaeli (2018), and use it to study the three-way tradeoff between rate, distortion, and perception. We show that restricting the perceptual quality to be high, generally leads to an elevation of the rate-distortion curve, thus necessitating a sacrifice in either rate or distortion. We prove several fundamental properties of this triple-tradeoff, calculate it in closed form for a Bernoulli source, and illustrate it visually on a toy MNIST example.

arXiv commentICML 2019 (long oral presentation), see talk at: https://slideslive.com/38917633/applications-computer-vision

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rényi Divergence and Kullback-Leibler Divergence
• Variational image compression with a scale hyperprior
• End-to-end Optimized Image Compression
• The Perception-Distortion Tradeoff
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Joint Autoregressive and Hierarchical Priors for Learned Image Compression
• Rate-Distortion-Perception Tradeoff of Variable-Length Source Coding for General Information Sources

Cited by in corpus (15)

• High-Fidelity Generative Image Compression
• Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks
• Better Compression with Deep Pre-Editing
• Information bottleneck through variational glasses
• Distributed Learning and Inference with Compressed Images
• On the advantages of stochastic encoders
• Learning to Compress Videos without Computing Motion
• Data Sharing and Compression for Cooperative Networked Control
• CBANet: Towards Complexity and Bitrate Adaptive Deep Image Compression using a Single Network
• A Rate-Distortion Framework for Characterizing Semantic Information
• CompressNet: Generative Compression at Extremely Low Bitrates
• Rate Distortion Theory for Descriptive Statistics
• Generative Adversarial User Privacy in Lossy Single-Server Information Retrieval
• Learned Image Compression for Machine Perception
• A Novel Framework for Image-to-image Translation and Image Compression