Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

Abstract

The most prominent problem associated with the deconvolution layer is the presence of checkerboard artifacts in output images and dense labels. To combat this problem, smoothness constraints, post processing and different architecture designs have been proposed. Odena et al. highlight three sources of checkerboard artifacts: deconvolution overlap, random initialization and loss functions. In this note, we proposed an initialization method for sub-pixel convolution known as convolution NN resize. Compared to sub-pixel convolution initialized with schemes designed for standard convolution kernels, it is free from checkerboard artifacts immediately after initialization. Compared to resize convolution, at the same computational complexity, it has more modelling power and converges to solutions with smaller test errors.

References in corpus (5)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Adversarially Learned Inference
• Is the deconvolution layer the same as a convolutional layer?

Cited by in corpus (26)

• Review: Deep Learning in Electron Microscopy
• InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
• Deep image prior for undersampling high-speed photoacoustic microscopy
• ExFuse: Enhancing Feature Fusion for Semantic Segmentation
• Lightweight Modules for Efficient Deep Learning based Image Restoration
• s-LWSR: Super Lightweight Super-Resolution Network
• DarkDeblur: Learning single-shot image deblurring in low-light condition
• Detecting Deepfake Videos: An Analysis of Three Techniques
• When Healthcare Meets Off-the-Shelf WiFi: A Non-Wearable and Low-Costs Approach for In-Home Monitoring
• Neuromorphologicaly-preserving Volumetric data encoding using VQ-VAE
• MXR-U-Nets for Real Time Hyperspectral Reconstruction
• Aerial Imagery Pixel-level Segmentation
• DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
• Super-Resolution using Convolutional Neural Networks without Any Checkerboard Artifacts
• Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Sounds
• On-Device Text Image Super Resolution
• Adversarial Augmentation for Enhancing Classification of Mammography Images
• CompressNet: Generative Compression at Extremely Low Bitrates
• Fixed smooth convolutional layer for avoiding checkerboard artifacts in CNNs
• Teaching deep neural networks to localize single molecules for super-resolution microscopy
• Many-to-Many Voice Conversion with Out-of-Dataset Speaker Support
• Subject-independent Human Pose Image Construction with Commodity Wi-Fi
• NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image
• Spectrally Consistent UNet for High Fidelity Image Transformations
• Holistic Decomposition Convolution for Effective Semantic Segmentation of 3D MR Images
• A Hybrid Approach Between Adversarial Generative Networks and Actor-Critic Policy Gradient for Low Rate High-Resolution Image Compression