Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
arXiv:2006.13560 · doi:10.1109/JSTSP.2020.3043590
Abstract
The past few years have witnessed increasing interests in applying deep learning to video compression. However, the existing approaches compress a video frame with only a few number of reference frames, which limits their ability to fully exploit the temporal correlation among video frames. To overcome this shortcoming, this paper proposes a Recurrent Learned Video Compression (RLVC) approach with the Recurrent Auto-Encoder (RAE) and Recurrent Probability Model (RPM). Specifically, the RAE employs recurrent cells in both the encoder and decoder. As such, the temporal information in a large range of frames can be used for generating latent representations and reconstructing compressed outputs. Furthermore, the proposed RPM network recurrently estimates the Probability Mass Function (PMF) of the latent representation, conditioned on the distribution of previous latent representations. Due to the correlation among consecutive frames, the conditional cross entropy can be lower than the independent cross entropy, thus reducing the bit-rate. The experiments show that our approach achieves the state-of-the-art learned video compression performance in terms of both PSNR and MS-SSIM. Moreover, our approach outperforms the default Low-Delay P (LDP) setting of x265 on PSNR, and also has better performance on MS-SSIM than the SSIM-tuned x265 and the slowest setting of x265. The codes are available at https://github.com/RenYang-home/RLVC.git.
arXiv commentAccepted for publication in IEEE Journal of Selected Topics in Signal Processing (J-STSP)
References in corpus (20)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Conditional Image Generation with PixelCNN Decoders
- Unsupervised Learning of Video Representations using LSTMs
- Video Enhancement with Task-Oriented Flow
- Variational image compression with a scale hyperprior
- End-to-end Optimized Image Compression
- Exploring the Limits of Language Modeling
- Visualizing and Understanding Recurrent Networks
- Medical image denoising using convolutional denoising autoencoders
- Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
- Joint Autoregressive and Hierarchical Priors for Learned Image Compression
- A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
- Reducing Complexity of HEVC: A Deep Learning Approach
- Context-adaptive Entropy Model for End-to-end Optimized Image Compression
- Video Compression With Rate-Distortion Autoencoders
- A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC
- Learning for Video Compression
- OpenDVC: An Open Source Implementation of the DVC Video Compression Method
Cited by in corpus (13)
- Temporal Context Mining for Learned Video Compression
- Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression
- Advancing Learned Video Compression with In-loop Frame Prediction
- Learned Video Compression via Heterogeneous Deformable Compensation Network
- Exploring Long- and Short-Range Temporal Information for Learned Video Compression
- OpenDVC: An Open Source Implementation of the DVC Video Compression Method
- Neighbor Correspondence Matching for Flow-based Video Frame Synthesis
- Survey on Visual Signal Coding and Processing with Generative Models: Technologies, Standards and Optimization
- FVC: A New Framework towards Deep Video Compression in Feature Space
- Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set
- Learned Wavelet Video Coding using Motion Compensated Temporal Filtering
- Versatile Learned Video Compression
- Content-Adaptive Inference for State-of-the-art Learned Video Compression