Lightweight Modules for Efficient Deep Learning based Image Restoration
arXiv:2007.05835 · doi:10.1109/TCSVT.2020.3007723
Abstract
Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the image-to-image' translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into image-to-image translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices.
Accepted at: IEEE Transactions on Circuits and Systems for Video Technology (Early Access Print) | |Codes Available at: https://github.com/avisekiit/TCSVT-LightWeight-CNNs | Supplementary Document at: https://drive.google.com/file/d/1BQhkh33Sen-d0qOrjq5h8ahw2VCUIVLg/view?usp=sharing
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- TensorFlow: A system for large-scale machine learning
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Compressing Neural Networks with the Hashing Trick
- Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network
- Semantic Segmentation using Adversarial Networks
- RENOIR - A Dataset for Real Low-Light Image Noise Reduction
- FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network
- Places: An Image Database for Deep Scene Understanding
- Real-world Noisy Image Denoising: A New Benchmark
- Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize
- Is the deconvolution layer the same as a convolutional layer?
- Efficient Deep Neural Network for Photo-realistic Image Super-Resolution