Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations

Abstract

We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy. Our method is based on a soft (continuous) relaxation of quantization and entropy, which we anneal to their discrete counterparts throughout training. We showcase this method for two challenging applications: Image compression and neural network compression. While these tasks have typically been approached with different methods, our soft-to-hard quantization approach gives results competitive with the state-of-the-art for both.

References in corpus (7)

• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Learning Structured Sparsity in Deep Neural Networks
• Lossy Image Compression with Compressive Autoencoders
• Is the deconvolution layer the same as a convolutional layer?
• Towards the Limit of Network Quantization
• Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks

Cited by in corpus (78)

• Variational image compression with a scale hyperprior
• Image and Video Compression with Neural Networks: A Review
• Recent Advances in Autoencoder-Based Representation Learning
• Joint Autoregressive and Hierarchical Priors for Learned Image Compression
• Neural Image Compression via Non-Local Attention Optimization and Improved Context Modeling
• Non-local Attention Optimized Deep Image Compression
• Video Compression With Rate-Distortion Autoencoders
• High-Fidelity Generative Image Compression
• Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
• Causal Contextual Prediction for Learned Image Compression
• Deep Learning-Based Video Coding: A Review and A Case Study
• Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
• Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
• Improving Inference for Neural Image Compression
• Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI
• Universally Quantized Neural Compression
• Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules
• Advancing Learned Video Compression with In-loop Frame Prediction
• Learning Accurate Entropy Model with Global Reference for Image Compression
• Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
• Psychoacoustic Calibration of Loss Functions for Efficient End-to-End Neural Audio Coding
• Improved Hybrid Layered Image Compression using Deep Learning and Traditional Codecs
• Learning Sparse Low-Precision Neural Networks With Learnable Regularization
• Scalable and Efficient Neural Speech Coding: A Hybrid Design
• Hierarchical Quantized Autoencoders
• LVQAC: Lattice Vector Quantization Coupled with Spatially Adaptive Companding for Efficient Learned Image Compression
• Latent-Domain Predictive Neural Speech Coding
• Generalized Octave Convolutions for Learned Multi-Frequency Image Compression
• Deep Perceptual Compression
• Video Compression through Image Interpolation
• Deep Generative Video Compression
• Deep Generative Models for Distribution-Preserving Lossy Compression
• End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video Compression
• Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding
• Disparity-based Stereo Image Compression with Aligned Cross-View Priors
• Learning to Compress Videos without Computing Motion
• Attention-Based Generative Neural Image Compression on Solar Dynamics Observatory
• Content Adaptive and Error Propagation Aware Deep Video Compression
• Gated Context Model with Embedded Priors for Deep Image Compression
• Multi-bin Trainable Linear Unit for Fast Image Restoration Networks
• Soft then Hard: Rethinking the Quantization in Neural Image Compression
• DSSLIC: Deep Semantic Segmentation-based Layered Image Compression
• On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework
• DVC: An End-to-end Deep Video Compression Framework
• Observer Dependent Lossy Image Compression
• BitNet: Bit-Regularized Deep Neural Networks
• Flexible Variable-Rate Image Feature Compression for Edge-Cloud Systems
• Machine Perception-Driven Image Compression: A Layered Generative Approach
• Scalar Quantization as Sparse Least Square Optimization
• Slimmable Compressive Autoencoders for Practical Neural Image Compression
• Learning a Single Tucker Decomposition Network for Lossy Image Compression with Multiple Bits-Per-Pixel Rates
• Compressing Colour Images with Joint Inpainting and Prediction
• Stochastic Layer-Wise Precision in Deep Neural Networks
• Learning End-to-End Lossy Image Compression: A Benchmark
• FALCON: Lightweight and Accurate Convolution
• Channel-Level Variable Quantization Network for Deep Image Compression
• Dissimilarity Mixture Autoencoder for Deep Clustering
• Learned Block-based Hybrid Image Compression
• DSIC: Deep Stereo Image Compression
• Learning to Structure an Image with Few Colors
• Learned Scalable Image Compression with Bidirectional Context Disentanglement Network
• A Robust Deep Learning-Based Beamforming Design for RIS-assisted Multiuser MISO Communications with Practical Constraints
• Learning Product Codebooks using Vector Quantized Autoencoders for Image Retrieval
• Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
• Neural Multi-scale Image Compression
• Learned Variable-Rate Image Compression with Residual Divisive Normalization
• Virtual Codec Supervised Re-Sampling Network for Image Compression
• Saliency Driven Perceptual Image Compression
• Variational Bayesian Quantization
• Neural Communication Systems with Bandwidth-limited Channel
• Out-of-Distribution Robustness in Deep Learning Compression
• Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compression
• Learned transform compression with optimized entropy encoding
• Source-Aware Neural Speech Coding for Noisy Speech Compression
• Quantization-Based Regularization for Autoencoders
• Progressive Spatial Recurrent Neural Network for Intra Prediction
• Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks
• Generative Memorize-Then-Recall framework for low bit-rate Surveillance Video Compression