Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model

Abstract

In recent years, neural network-driven image compression (NIC) has gained significant attention. Some works adopt deep generative models such as GANs and diffusion models to enhance perceptual quality (realism). A critical obstacle of these generative NIC methods is that each model is optimized for a single bit rate. Consequently, multiple models are required to compress images to different bit rates, which is impractical for real-world applications. To tackle this issue, we propose a variable-rate generative NIC model. Specifically, we explore several discriminator designs tailored for the variable-rate approach and introduce a novel adversarial loss. Moreover, by incorporating the newly proposed multi-realism technique, our method allows the users to adjust the bit rate, distortion, and realism with a single model, achieving ultra-controllability. Unlike existing variable-rate generative NIC models, our method matches or surpasses the performance of state-of-the-art single-rate generative NIC models while covering a wide range of bit rates using just one model. Code will be available at https://github.com/iwa-shi/CRDR

WACV2024 Oral. Code is at https://github.com/iwa-shi/CRDR

References in corpus (17)

• Denoising Diffusion Probabilistic Models
• Variational image compression with a scale hyperprior
• End-to-end Optimized Image Compression
• The Perception-Distortion Tradeoff
• Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
• Non-local Attention Optimized Deep Image Compression
• High-Fidelity Generative Image Compression
• Variable Rate Deep Image Compression with Modulated Autoencoder
• Contextformer: A Transformer with Spatio-Channel Attention for Context Modeling in Learned Image Compression
• Lossy Image Compression with Conditional Diffusion Models
• Lossy Compression with Gaussian Diffusion
• Interpolation variable rate image compression
• High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation
• Image Compression with Product Quantized Masked Image Modeling
• A Residual Diffusion Model for High Perceptual Quality Codec Augmentation
• High-Fidelity Image Compression with Score-based Generative Models
• Flexible Neural Image Compression via Code Editing