High-Fidelity Generative Image Compression

Abstract

We extensively study how to combine Generative Adversarial Networks and learned compression to obtain a state-of-the-art generative lossy compression system. In particular, we investigate normalization layers, generator and discriminator architectures, training strategies, as well as perceptual losses. In contrast to previous work, i) we obtain visually pleasing reconstructions that are perceptually similar to the input, ii) we operate in a broad range of bitrates, and iii) our approach can be applied to high-resolution images. We bridge the gap between rate-distortion-perception theory and practice by evaluating our approach both quantitatively with various perceptual metrics, and with a user study. The study shows that our method is preferred to previous approaches even if they use more than 2x the bitrate.

arXiv commentThis is the Camera Ready version for NeurIPS 2020. Project page: https://hific.github.io

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Spectral Normalization for Generative Adversarial Networks
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Variational image compression with a scale hyperprior
• End-to-end Optimized Image Compression
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Joint Autoregressive and Hierarchical Priors for Learned Image Compression
• Context-adaptive Entropy Model for End-to-end Optimized Image Compression
• Variable Rate Image Compression with Recurrent Neural Networks
• Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
• Learning Convolutional Networks for Content-weighted Image Compression

Cited by in corpus (17)

• Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning
• Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
• Boosting Neural Image Compression for Machines Using Latent Space Masking
• Applying Convolutional Neural Networks to Data on Unstructured Meshes with Space-Filling Curves
• Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model
• High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation
• On Quantizing Implicit Neural Representations
• Analysis of Neural Image Compression Networks for Machine-to-Machine Communication
• Disparity-based Stereo Image Compression with Aligned Cross-View Priors
• Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory
• Toward Scalable Image Feature Compression: A Content-Adaptive and Diffusion-Based Approach
• Bridging the gap between image coding for machines and humans
• Machine Perception-Driven Image Compression: A Layered Generative Approach
• Transform and Bitstream Domain Image Classification
• Evaluating the Stability of Deep Image Quality Assessment With Respect to Image Scaling
• ABO: Abandon Bayer Filter for Adaptive Edge Offloading in Responsive Augmented Reality
• Learned Image Compression with Separate Hyperprior Decoders