Variational image compression with a scale hyperprior
arXiv:1802.01436
Abstract
We describe an end-to-end trainable model for image compression based on variational autoencoders. The model incorporates a hyperprior to effectively capture spatial dependencies in the latent representation. This hyperprior relates to side information, a concept universal to virtually all modern image codecs, but largely unexplored in image compression using artificial neural networks (ANNs). Unlike existing autoencoder compression methods, our model trains a complex prior jointly with the underlying autoencoder. We demonstrate that this model leads to state-of-the-art image compression when measuring visual quality using the popular MS-SSIM index, and yields rate-distortion performance surpassing published ANN-based methods when evaluated using a more traditional metric based on squared error (PSNR). Furthermore, we provide a qualitative comparison of models trained for different distortion metrics.
accepted as a conference contribution to International Conference on Learning Representations 2018
References in corpus (3)
Cited by in corpus (155)
- Image and Video Compression with Neural Networks: A Review
- Joint Autoregressive and Hierarchical Priors for Learned Image Compression
- Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
- Non-local Attention Optimized Deep Image Compression
- Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
- Learned Point Cloud Geometry Compression
- Temporal Context Mining for Learned Video Compression
- Video Compression With Rate-Distortion Autoencoders
- High-Fidelity Generative Image Compression
- Learning Convolutional Transforms for Lossy Point Cloud Geometry Compression
- Causal Contextual Prediction for Learned Image Compression
- Scalable Image Coding for Humans and Machines
- Deep Learning-Based Video Coding: A Review and A Case Study
- Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
- Image coding for machines: an end-to-end learned approach
- Variable Rate Deep Image Compression with Modulated Autoencoder
- QARV: Quantization-Aware ResNet VAE for Lossy Image Compression
- Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression
- Supervised Compression for Resource-Constrained Edge Computing Systems
- Non-Orthogonal Multiple Access Enhanced Multi-User Semantic Communication
- ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression
- A Unified End-to-End Framework for Efficient Deep Image Compression
- Deep Joint Source-Channel Coding for Wireless Image Retrieval
- Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
- Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds
- Improving Inference for Neural Image Compression
- Lossy Image Compression with Quantized Hierarchical VAEs
- Attention-based Convolutional Autoencoders for 3D-Variational Data Assimilation
- Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks
- Universally Quantized Neural Compression
- Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules
- Advancing Learned Video Compression with In-loop Frame Prediction
- Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning
- Learning based Facial Image Compression with Semantic Fidelity Metric
- An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization
- End-to-End Learnable Multi-Scale Feature Compression for VCM
- Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
- OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression
- Better Compression with Deep Pre-Editing
- Improved Hybrid Layered Image Compression using Deep Learning and Traditional Codecs
- Boosting Neural Image Compression for Machines Using Latent Space Masking
- Hierarchical Quantized Autoencoders
- LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression
- COIN: COmpression with Implicit Neural representations
- Learned Video Compression via Heterogeneous Deformable Compensation Network
- The Bjøntegaard Bible -- Why your Way of Comparing Video Codecs May Be Wrong
- Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks
- Generalized Octave Convolutions for Learned Multi-Frequency Image Compression
- Deep Perceptual Compression
- GroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compression
- Transform Network Architectures for Deep Learning based End-to-End Image/Video Coding in Subsampled Color Spaces
- Human Perceptual Evaluations for Image Compression
- Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks
- Deep Generative Video Compression
- Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
- Geometric Prior Based Deep Human Point Cloud Geometry Compression
- FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
- Controlling Rate, Distortion, and Realism: Towards a Single Comprehensive Neural Image Compression Model
- Deep Generative Models for Distribution-Preserving Lossy Compression
- OMR-NET: a two-stage octave multi-scale residual network for screen content image compression
- Exploring Long- and Short-Range Temporal Information for Learned Video Compression
- End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression
- Speeding up VP9 Intra Encoder with Hierarchical Deep Learning Based Partition Prediction
- Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics
- FOOL: Addressing the Downlink Bottleneck in Satellite Computing with Neural Feature Compression
- V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
- OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression
- Distributed Learning and Inference with Compressed Images
- Image compression optimized for 3D reconstruction by utilizing deep neural networks
- Generalized Gaussian Model for Learned Image Compression
- Unified learning-based lossy and lossless JPEG recompression
- BINet: a binary inpainting network for deep patch-based image compression
- Frequency Disentangled Features in Neural Image Compression
- Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization
- LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression
- A Cross Channel Context Model for Latents in Deep Image Compression
- Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression
- Immersive Video Compression using Implicit Neural Representations
- Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory
- The JPEG Pleno Learning-based Point Cloud Coding Standard: Serving Man and Machine
- Neural Video Compression using Spatio-Temporal Priors
- Learned Scalable Video Coding For Humans and Machines
- Analysis of Neural Image Compression Networks for Machine-to-Machine Communication
- Progressive Neural Compression for Adaptive Image Offloading under Timing Constraints
- Recognition-Aware Learned Image Compression
- IDLat: An Importance-Driven Latent Generation Method for Scientific Data
- Neural Distributed Compressor Discovers Binning
- Content Adaptive and Error Propagation Aware Deep Video Compression
- Learned Wavelet Video Coding using Motion Compensated Temporal Filtering
- On the Importance of Denoising when Learning to Compress Images
- Comprehensive Complexity Assessment of Emerging Learned Image Compression on CPU and GPU
- Gated Context Model with Embedded Priors for Deep Image Compression
- Scalable Model Compression by Entropy Penalized Reparameterization
- DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedback
- Color Learning for Image Compression
- End-to-end optimized image compression with competition of prior distributions
- End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement
- Neural-based Compression Scheme for Solar Image Data
- Learning-Based Conditional Image Coder Using Color Separation
- DVC: An End-to-end Deep Video Compression Framework
- Towards Empirical Sandwich Bounds on the Rate-Distortion Function
- RoVISQ: Reduction of Video Service Quality via Adversarial Attacks on Deep Learning-based Video Compression
- Differentiable bit-rate estimation for neural-based video codec enhancement
- Channel-wise Feature Decorrelation for Enhanced Learned Image Compression
- Versatile Learned Video Compression
- L3C-Stereo: Lossless Compression for Stereo Images
- Learned Variable-Rate Multi-Frequency Image Compression using Modulated Generalized Octave Convolution
- Slimmable Encoders for Flexible Split DNNs in Bandwidth and Resource Constrained IoT Systems
- The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study
- The State of Applying Artificial Intelligence to Tissue Imaging for Cancer Research and Early Detection
- 3-D Context Entropy Model for Improved Practical Image Compression
- End-to-end Optimized Video Compression with MV-Residual Prediction
- Learned Video Compression with Feature-level Residuals
- Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip
- Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data
- Learning End-to-End Lossy Image Compression: A Benchmark
- Extreme Image Coding via Multiscale Autoencoders With Generative Adversarial Optimization
- End-to-End Learning for Video Frame Compression with Self-Attention
- CAESR: Conditional Autoencoder and Super-Resolution for Learned Spatial Scalability
- Learned Scalable Image Compression with Bidirectional Context Disentanglement Network
- Towards Analysis-friendly Face Representation with Scalable Feature and Texture Compression
- DSIC: Deep Stereo Image Compression
- Binocular Rivalry Oriented Predictive Auto-Encoding Network for Blind Stereoscopic Image Quality Measurement
- Convolutional Transformer-Based Image Compression
- Invertible Image Rescaling
- Saliency Driven Perceptual Image Compression
- Object-Based Image Coding: A Learning-Driven Revisit
- Neural Communication Systems with Bandwidth-limited Channel
- Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
- Compressing Sign Information in DCT-based Image Coding via Deep Sign Retrieval
- Multiscale Augmented Normalizing Flows for Image Compression
- Region of Interest Loss for Anonymizing Learned Image Compression
- Optimized learned entropy coding parameters for practical neural-based image and video compression
- Substitutional Neural Image Compression
- End-to-End Image Compression with Probabilistic Decoding
- Learned Video Codec with Enriched Reconstruction for CLIC P-frame Coding
- Attention-guided Image Compression by Deep Reconstruction of Compressive Sensed Saliency Skeleton
- Visual Analysis Motivated Rate-Distortion Model for Image Coding
- Leveraging Domain Knowledge using Machine Learning for Image Compression in Internet-of-Things
- Sibling Neural Estimators: Improving Iterative Image Decoding with Gradient Communication
- Variational Bayesian Quantization
- Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks
- Perceptual Quality Assessment of Omnidirectional Images as Moving Camera Videos
- Object Detection-Based Variable Quantization Processing
- Deep motion estimation for parallel inter-frame prediction in video compression
- Learned Image Compression with Soft Bit-based Rate-Distortion Optimization
- Generative Memorize-Then-Recall framework for low bit-rate Surveillance Video Compression
- Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization
- Compressive Sampling for Array Cameras
- Deep Clustering of Compressed Variational Embeddings
- Progressive Compressed Records: Taking a Byte out of Deep Learning Data
- Variable Rate Image Compression Method with Dead-zone Quantizer
- DeepCompress: Efficient Point Cloud Geometry Compression
- Neural Image Decompression: Learning to Render Better Image Previews
- Learning to Learn to Compress