MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering
arXiv:2007.07099 · doi:10.1109/JSTSP.2020.3043064
Abstract
In this paper, we propose a novel convolutional neural network (CNN) architecture, MFRNet, for post-processing (PP) and in-loop filtering (ILF) in the context of video compression. This network consists of four Multi-level Feature review Residual dense Blocks (MFRBs), which are connected using a cascading structure. Each MFRB extracts features from multiple convolutional layers using dense connections and a multi-level residual learning structure. In order to further improve information flow between these blocks, each of them also reuses high dimensional features from the previous MFRB. This network has been integrated into PP and ILF coding modules for both HEVC (HM 16.20) and VVC (VTM 7.0), and fully evaluated under the JVET Common Test Conditions using the Random Access configuration. The experimental results show significant and consistent coding gains over both anchor codecs (HEVC HM and VVC VTM) and also over other existing CNN-based PP/ILF approaches based on Bjontegaard Delta measurements using both PSNR and VMAF for quality assessment. When MFRNet is integrated into HM 16.20, gains up to 16.0% (BD-rate VMAF) are demonstrated for ILF, and up to 21.0% (BD-rate VMAF) for PP. The respective gains for VTM 7.0 are up to 5.1% for ILF and up to 7.1% for PP.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- U-Net: Going Deeper with Nested U-Structure for Salient Object Detection
- End-to-end Optimized Image Compression
- Training Very Deep Networks
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
- Highway Networks
- Image and Video Compression with Neural Networks: A Review
- A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
- Video Compression With Rate-Distortion Autoencoders
- Deep Learning-Based Video Coding: A Review and A Case Study
- Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC
- Neural network-based arithmetic coding of intra prediction modes in HEVC
- ViSTRA2: Video Coding using Spatial Resolution and Effective Bit Depth Adaptation
- Efficient Deep Neural Network for Photo-realistic Image Super-Resolution
Cited by in corpus (11)
- BVI-DVC: A Training Database for Deep Video Compression
- CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement
- Immersive Video Compression using Implicit Neural Representations
- Compressing Deep Image Super-resolution Models
- BVI-AOM: A New Training Dataset for Deep Video Compression Optimization
- Revisiting the Sample Adaptive Offset post-filter of VVC with Neural-Networks
- Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillation
- RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content
- DaBiT: Depth and Blur informed Transformer for Video Focal Deblurring
- Enhancing HDR Video Compression through CNN-based Effective Bit Depth Adaptation
- ViSTRA3: Video Coding with Deep Parameter Adaptation and Post Processing