Generalized Octave Convolutions for Learned Multi-Frequency Image Compression

Abstract

Learned image compression has recently shown the potential to outperform the standard codecs. State-of-the-art rate-distortion (R-D) performance has been achieved by context-adaptive entropy coding approaches in which hyperprior and autoregressive models are jointly utilized to effectively capture the spatial dependencies in the latent representations. However, the latents are feature maps of the same spatial resolution in previous works, which contain some redundancies that affect the R-D performance. In this paper, we propose the first learned multi-frequency image compression and entropy coding approach that is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components, where the low frequency is represented by a lower resolution. Therefore, its spatial redundancy is reduced, which improves the R-D performance. Novel generalized octave convolution and octave transposed-convolution architectures with internal activation layers are also proposed to preserve more spatial structure of the information. Experimental results show that the proposed scheme not only outperforms all existing learned methods as well as standard codecs such as the next-generation video coding standard VVC (4:2:0) on the Kodak dataset in both PSNR and MS-SSIM. We also show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks such as semantic segmentation and image denoising.

arXiv comment13 pages, 10 figures, 5 tables; Extended version of the paper accepted to AAAI 2021

References in corpus (13)

• Striving for Simplicity: The All Convolutional Net
• Conditional Image Generation with PixelCNN Decoders
• Variational image compression with a scale hyperprior
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Joint Autoregressive and Hierarchical Priors for Learned Image Compression
• Context-adaptive Entropy Model for End-to-end Optimized Image Compression
• Lossy Image Compression with Compressive Autoencoders
• Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
• Variable Rate Image Compression with Recurrent Neural Networks
• An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization
• Deep Learning on Image Denoising: An overview
• Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks
• Learning Content-Weighted Deep Image Compression

Cited by in corpus (4)

• Recognition-Aware Learned Image Compression
• Learned Variable-Rate Multi-Frequency Image Compression using Modulated Generalized Octave Convolution
• Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution
• Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks