Advancing Learned Video Compression with In-loop Frame Prediction
arXiv:2211.07004 · doi:10.1109/TCSVT.2022.3222418
Abstract
Recent years have witnessed an increasing interest in end-to-end learned video compression. Most previous works explore temporal redundancy by detecting and compressing a motion map to warp the reference frame towards the target frame. Yet, it failed to adequately take advantage of the historical priors in the sequential reference frames. In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module, which is able to effectively predict the target frame from the previously compressed frames, without consuming any bit-rate. The predicted frame can serve as a better reference than the previously compressed frame, and therefore it benefits the compression performance. The proposed in-loop prediction module is a part of the end-to-end video compression and is jointly optimized in the whole framework. We propose the recurrent and the bi-directional in-loop prediction modules for compressing P-frames and B-frames, respectively. The experiments show the state-of-the-art performance of our ALVC approach in learned video compression. We also outperform the default hierarchical B mode of x265 in terms of PSNR and beat the slowest mode of the SSIM-tuned x265 on MS-SSIM. The project page: https://github.com/RenYang-home/ALVC.
References in corpus (15)
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
- Video Enhancement with Task-Oriented Flow
- Variational image compression with a scale hyperprior
- End-to-end Optimized Image Compression
- Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
- PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
- Context-adaptive Entropy Model for End-to-end Optimized Image Compression
- Video Compression With Rate-Distortion Autoencoders
- M-LVC: Multiple Frames Prediction for Learned Video Compression
- Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
- Deep Contextual Video Compression
- Convolutional Tensor-Train LSTM for Spatio-temporal Learning
- OpenDVC: An Open Source Implementation of the DVC Video Compression Method
- Perceptual Learned Video Compression with Recurrent Conditional GAN
- End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation