Learning for Video Compression
arXiv:1804.09869 · doi:10.1109/TCSVT.2019.2892608
Abstract
One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis, binarization, etc. Experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.
arXiv commentAccepted by IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Pixel Recurrent Neural Networks
- End-to-end Optimized Image Compression
- A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
- Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding
- Real-Time Adaptive Image Compression
- Towards Conceptual Compression
- Neural network-based arithmetic coding of intra prediction modes in HEVC
- A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding
- Learning based Facial Image Compression with Semantic Fidelity Metric
- Recurrent Spatial Transformer Networks
- Techniques for Learning Binary Stochastic Feedforward Neural Networks
- Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks
- Generative Compression
Cited by in corpus (30)
- Image and Video Compression with Neural Networks: A Review
- Temporal Context Mining for Learned Video Compression
- Video Compression With Rate-Distortion Autoencoders
- Deep Learning-Based Video Coding: A Review and A Case Study
- Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
- Learning based Facial Image Compression with Semantic Fidelity Metric
- Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression
- Recent Advances on HEVC Inter-frame Coding: From Optimization to Implementation and Beyond
- Deep Generative Video Compression
- Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization
- Learning to Compress Videos without Computing Motion
- Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set
- Content Adaptive and Error Propagation Aware Deep Video Compression
- HPC: Hierarchical Progressive Coding Framework for Volumetric Video
- Hierarchical Autoregressive Modeling for Neural Video Compression
- DVC: An End-to-end Deep Video Compression Framework
- Versatile Learned Video Compression
- Improving Deep Video Compression by Resolution-adaptive Flow Coding
- Can Learned Frame-Prediction Compete with Block-Motion Compensation for Video Coding?
- Learned Block-based Hybrid Image Compression
- Learned Video Compression with Feature-level Residuals
- End-to-end Optimized Video Compression with MV-Residual Prediction
- Variable Rate Video Compression using a Hybrid Recurrent Convolutional Learning Framework
- End-to-End Learning for Video Frame Compression with Self-Attention
- Learned Video Compression with Residual Prediction and Loop Filter
- Neural Video Compression with In-Loop Contextual Filtering and Out-of-Loop Reconstruction Enhancement
- Generative Memorize-Then-Recall framework for low bit-rate Surveillance Video Compression
- A Foreground-background Parallel Compression with Residual Encoding for Surveillance Video
- Deep motion estimation for parallel inter-frame prediction in video compression
- Neural Generation of Blocks for Video Coding