End-to-end Optimized Image Compression
arXiv:1611.01704
Abstract
We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.
Published as a conference paper at ICLR 2017
References in corpus (2)
Cited by in corpus (171)
- Deep Joint Source-Channel Coding for Wireless Image Transmission
- Image Quality Assessment: Unifying Structure and Texture Similarity
- Wireless Image Transmission Using Deep Source Channel Coding With Attention Modules
- Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
- Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
- Non-local Attention Optimized Deep Image Compression
- Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
- Learned Point Cloud Geometry Compression
- Video Compression With Rate-Distortion Autoencoders
- Comparison of Image Quality Models for Optimization of Image Processing Systems
- High-Fidelity Generative Image Compression
- Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
- Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression
- Causal Contextual Prediction for Learned Image Compression
- BVI-DVC: A Training Database for Deep Video Compression
- Deep Learning-Based Video Coding: A Review and A Case Study
- Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
- Learning for Video Compression
- Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
- Deep Contextual Video Compression
- Variable Rate Deep Image Compression with Modulated Autoencoder
- Real-Time Adaptive Image Compression
- From Variational to Deterministic Autoencoders
- Joint Device-Edge Inference over Wireless Links with Pruning
- QARV: Quantization-Aware ResNet VAE for Lossy Image Compression
- Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression
- Supervised Compression for Resource-Constrained Edge Computing Systems
- MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering
- An analysis on the use of autoencoders for representation learning: fundamentals, learning task case studies, explainability and challenges
- Interpretable Detail-Fidelity Attention Network for Single Image Super-Resolution
- A DNA Based Colour Image Encryption Scheme Using A Convolutional Autoencoder
- ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression
- A Unified End-to-End Framework for Efficient Deep Image Compression
- Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
- Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds
- Improving Inference for Neural Image Compression
- On the Role of ViT and CNN in Semantic Communications: Analysis and Prototype Validation
- Video Compression with CNN-based Post Processing
- Universally Quantized Neural Compression
- Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks
- Advancing Learned Video Compression with In-loop Frame Prediction
- Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning
- Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules
- Derivatives and Inverse of Cascaded Linear+Nonlinear Neural Models
- Learning based Facial Image Compression with Semantic Fidelity Metric
- An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization
- Fixing a Broken ELBO
- End-to-End Learnable Multi-Scale Feature Compression for VCM
- Learning to Inpaint for Image Compression
- Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
- Quality Prediction on Deep Generative Images
- OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression
- Better Compression with Deep Pre-Editing
- Improved Hybrid Layered Image Compression using Deep Learning and Traditional Codecs
- Machine Learning Techniques to Construct Patched Analog Ensembles for Data Assimilation
- Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks
- Boosting Neural Image Compression for Machines Using Latent Space Masking
- Learned Video Compression via Heterogeneous Deformable Compensation Network
- BlockCNN: A Deep Network for Artifact Removal and Image Compression
- Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks
- Computationally Efficient Neural Image Compression
- Video Compression through Image Interpolation
- Deep Perceptual Compression
- Deep Generative Video Compression
- Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks
- Transform Network Architectures for Deep Learning based End-to-End Image/Video Coding in Subsampled Color Spaces
- Eigen-Distortions of Hierarchical Representations
- Learning Convolutional Networks for Content-weighted Image Compression
- Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding
- Human Perceptual Evaluations for Image Compression
- Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model
- FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
- Geometric Prior Based Deep Human Point Cloud Geometry Compression
- Towards Metamerism via Foveated Style Transfer
- Unsupervised Out-of-Distribution Detection with Batch Normalization
- Potential of deep features for opinion-unaware, distortion-unaware, no-reference image quality assessment
- OMR-NET: a two-stage octave multi-scale residual network for screen content image compression
- Biased Mixtures Of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations
- OpenDVC: An Open Source Implementation of the DVC Video Compression Method
- Exploring Long- and Short-Range Temporal Information for Learned Video Compression
- Deep Generative Models for Distribution-Preserving Lossy Compression
- Distributed Learning and Inference with Compressed Images
- FOOL: Addressing the Downlink Bottleneck in Satellite Computing with Neural Feature Compression
- Image compression optimized for 3D reconstruction by utilizing deep neural networks
- End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression
- Unified learning-based lossy and lossless JPEG recompression
- BINet: a binary inpainting network for deep patch-based image compression
- A Cross Channel Context Model for Latents in Deep Image Compression
- Rate-Distortion-Perception Tradeoff for Gaussian Vector Sources
- LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression
- Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization
- Immersive Video Compression using Implicit Neural Representations
- Adaptive Denoising via GainTuning
- Learning to Compress Videos without Computing Motion
- Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory
- Analysis of Neural Image Compression Networks for Machine-to-Machine Communication
- On the Importance of Denoising when Learning to Compress Images
- Neural Distributed Compressor Discovers Binning
- Learned Wavelet Video Coding using Motion Compensated Temporal Filtering
- IDLat: An Importance-Driven Latent Generation Method for Scientific Data
- Content Adaptive and Error Propagation Aware Deep Video Compression
- Variational Bayes image restoration with compressive autoencoders
- DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback
- Scalable Model Compression by Entropy Penalized Reparameterization
- Gated Context Model with Embedded Priors for Deep Image Compression
- End-to-end optimized image compression with competition of prior distributions
- Be Like Water: Robustness to Extraneous Variables Via Adaptive Feature Normalization
- End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement
- DSSLIC: Deep Semantic Segmentation-based Layered Image Compression
- Predictive Sampling with Forecasting Autoregressive Models
- Neural-based Compression Scheme for Solar Image Data
- Learning-Based Conditional Image Coder Using Color Separation
- Observer Dependent Lossy Image Compression
- Adversarial Video Compression Guided by Soft Edge Detection
- RoVISQ: Reduction of Video Service Quality via Adversarial Attacks on Deep Learning-based Video Compression
- Machine Perception-Driven Image Compression: A Layered Generative Approach
- Channel-wise Feature Decorrelation for Enhanced Learned Image Compression
- L3C-Stereo: Lossless Compression for Stereo Images
- Versatile Learned Video Compression
- Active Fine-Tuning from gMAD Examples Improves Blind Image Quality Assessment
- Flexible Variable-Rate Image Feature Compression for Edge-Cloud Systems
- Learning a Single Tucker Decomposition Network for Lossy Image Compression with Multiple Bits-Per-Pixel Rates
- Single-Training Collaborative Object Detectors Adaptive to Bandwidth and Computation
- Visual Information flow in Wilson-Cowan networks
- Slimmable Compressive Autoencoders for Practical Neural Image Compression
- Slimmable Encoders for Flexible Split DNNs in Bandwidth and Resource Constrained IoT Systems
- CompressNet: Generative Compression at Extremely Low Bitrates
- Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data
- Learning End-to-End Lossy Image Compression: A Benchmark
- Learning Frequency-Specific Quantization Scaling in VVC for Standard-Compliant Task-driven Image Coding
- Extreme Image Coding via Multiscale Autoencoders With Generative Adversarial Optimization
- Learning-based Compression for Material and Texture Recognition
- 3-D Context Entropy Model for Improved Practical Image Compression
- Learned Video Compression with Feature-level Residuals
- CAESR: Conditional Autoencoder and Super-Resolution for Learned Spatial Scalability
- Neural Image Compression and Explanation
- Learning to Structure an Image with Few Colors
- Invertible Image Rescaling
- Towards Analysis-friendly Face Representation with Scalable Feature and Texture Compression
- Learning Product Codebooks using Vector Quantized Autoencoders for Image Retrieval
- Neural Multi-scale Image Compression
- Convolutional Transformer-Based Image Compression
- Binocular Rivalry Oriented Predictive Auto-Encoding Network for Blind Stereoscopic Image Quality Measurement
- LDC-VAE: A Latent Distribution Consistency Approach to Variational AutoEncoders
- Compressing Sign Information in DCT-based Image Coding via Deep Sign Retrieval
- DeepSIC: Deep Semantic Image Compression
- Scalable Facial Image Compression with Deep Feature Reconstruction
- Unified Signal Compression Using Generative Adversarial Networks
- Region of Interest Loss for Anonymizing Learned Image Compression
- Substitutional Neural Image Compression
- Variational Bayesian Quantization
- Saliency Driven Perceptual Image Compression
- End-to-End Image Compression with Probabilistic Decoding
- Deep Learning-based Image Compression with Trellis Coded Quantization
- Object-Based Image Coding: A Learning-Driven Revisit
- Exploring Autoencoder-based Error-bounded Compression for Scientific Data
- Attention-guided Image Compression by Deep Reconstruction of Compressive Sensed Saliency Skeleton
- Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
- Multiscale Augmented Normalizing Flows for Image Compression
- Quantization-Based Regularization for Autoencoders
- Place-specific Background Modeling Using Recursive Autoencoders
- Learned Image Compression with Soft Bit-based Rate-Distortion Optimization
- Investigating Image Applications Based on Spatial-Frequency Transform and Deep Learning Techniques
- Learning to Learn to Compress
- Towards improved lossy image compression: Human image reconstruction with public-domain images
- Multi-scale Grouped Dense Network for VVC Intra Coding
- Generative Memorize-Then-Recall framework for low bit-rate Surveillance Video Compression
- Progressive Spatial Recurrent Neural Network for Intra Prediction
- Customized OCT images compression scheme with deep neural network
- Towards Modality Transferable Visual Information Representation with Optimal Model Compression
- Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks