Image and Video Compression with Neural Networks: A Review
arXiv:1904.03567 · doi:10.1109/TCSVT.2019.2910119
Abstract
In recent years, the image and video coding technologies have advanced by leaps and bounds. However, due to the popularization of image and video acquisition devices, the growth rate of image and video data is far beyond the improvement of the compression ratio. In particular, it has been widely recognized that there are increasing challenges of pursuing further coding performance improvement within the traditional hybrid coding framework. Deep convolution neural network (CNN) which makes the neural network resurge in recent years and has achieved great success in both artificial intelligent and signal processing fields, also provides a novel and promising solution for image and video compression. In this paper, we provide a systematic, comprehensive and up-to-date review of neural network based image and video compression techniques. The evolution and development of neural network based compression methodologies are introduced for images and video respectively. More specifically, the cutting-edge video coding techniques by leveraging deep learning and HEVC framework are presented and discussed, which promote the state-of-the-art video coding performance substantially. Moreover, the end-to-end image and video coding frameworks based on neural networks are also reviewed, revealing interesting explorations on next generation image and video coding frameworks/standards. The most significant research works on the image and video coding related topics using neural networks are highlighted, and future trends are also envisioned. In particular, the joint compression on semantic and visual information is tentatively explored to formulate high efficiency signal representation structure for both human vision and machine vision, which are the two dominant signal receptor in the age of artificial intelligence.
arXiv commentAccepted by IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT) as transactions paper
References in corpus (21)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Unsupervised Learning of Video Representations using LSTMs
- Variational image compression with a scale hyperprior
- DRAW: A Recurrent Neural Network For Image Generation
- Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
- Joint Autoregressive and Hierarchical Priors for Learned Image Compression
- A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
- Reducing Complexity of HEVC: A Deep Learning Approach
- Video (language) modeling: a baseline for generative models of natural videos
- Lossy Image Compression with Compressive Autoencoders
- Multi-Frame Quality Enhancement for Compressed Video
- CAS-CNN: A Deep Convolutional Neural Network for Image Compression Artifact Suppression
- Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding
- Learning for Video Compression
- Real-Time Adaptive Image Compression
- Towards Conceptual Compression
- Video Frame Interpolation via Adaptive Separable Convolution
- Spatial-Temporal Residue Network Based In-Loop Filter for Video Coding
- Neural network-based arithmetic coding of intra prediction modes in HEVC
- S-Net: A Scalable Convolutional Neural Network for JPEG Compression Artifact Reduction
- A practical convolutional neural network as loop filter for intra frame
Cited by in corpus (31)
- Artificial Intelligence in the Creative Industries: A Review
- Temporal Context Mining for Learned Video Compression
- BVI-DVC: A Training Database for Deep Video Compression
- Deep Learning-Based Video Coding: A Review and A Case Study
- Review: Deep Learning in Electron Microscopy
- MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering
- Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuning
- ViSTRA2: Video Coding using Spatial Resolution and Effective Bit Depth Adaptation
- Recent Advances on HEVC Inter-frame Coding: From Optimization to Implementation and Beyond
- Hierarchical Quantized Autoencoders
- Cross Modal Compression: Towards Human-comprehensible Semantic Compression
- CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement
- Rapid training of quantum recurrent neural networks
- Modeling Lost Information in Lossy Image Compression
- Interpreting CNN for Low Complexity Learned Sub-pixel Motion Compensation in Video Coding
- ECVC: Exploiting Non-Local Correlations in Multiple Frames for Contextual Video Compression
- LLIC: Large Receptive Field Transform Coding with Adaptive Weights for Learned Image Compression
- Comprehensive Complexity Assessment of Emerging Learned Image Compression on CPU and GPU
- End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement
- Differentiable bit-rate estimation for neural-based video codec enhancement
- End-to-End Learned Image Compression with Quantized Weights and Activations
- Machine Perception-Driven Image Compression: A Layered Generative Approach
- Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model
- Towards Analysis-friendly Face Representation with Scalable Feature and Texture Compression
- Optimized learned entropy coding parameters for practical neural-based image and video compression
- Atom: Neural Traffic Compression with Spatio-Temporal Graph Neural Networks
- Neural Communication Systems with Bandwidth-limited Channel
- Learned Video Compression with Residual Prediction and Loop Filter
- Sub-band coding of hexagonal images
- Neural Weight Step Video Compression
- Neural Generation of Blocks for Video Coding