Temporal Context Mining for Learned Video Compression
arXiv:2111.13850 · doi:10.1109/TMM.2022.3220421
Abstract
We address end-to-end learned video compression with a special focus on better learning and utilizing temporal contexts. For temporal context mining, we propose to store not only the previously reconstructed frames, but also the propagated features into the generalized decoded picture buffer. From the stored propagated features, we propose to learn multi-scale temporal contexts, and re-fill the learned temporal contexts into the modules of our compression scheme, including the contextual encoder-decoder, the frame generator, and the temporal context encoder. Our scheme discards the parallelization-unfriendly auto-regressive entropy model to pursue a more practical decoding time. We compare our scheme with x264 and x265 (representing industrial software for H.264 and H.265, respectively) as well as the official reference software for H.264, H.265, and H.266 (JM, HM, and VTM, respectively). When intra period is 32 and oriented to PSNR, our scheme outperforms H.265--HM by 14.4% bit rate saving; when oriented to MS-SSIM, our scheme outperforms H.266--VTM by 21.1% bit rate saving.
References in corpus (12)
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Video Enhancement with Task-Oriented Flow
- Variational image compression with a scale hyperprior
- Image and Video Compression with Neural Networks: A Review
- CompressAI: a PyTorch library and evaluation platform for end-to-end compression research
- Video Compression With Rate-Distortion Autoencoders
- M-LVC: Multiple Frames Prediction for Learned Video Compression
- Causal Contextual Prediction for Learned Image Compression
- BVI-DVC: A Training Database for Deep Video Compression
- Learning for Video Compression with Recurrent Auto-Encoder and Recurrent Probability Model
- Learning for Video Compression
- Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression