Is the deconvolution layer the same as a convolutional layer?

Abstract

In this note, we want to focus on aspects related to two questions most people asked us at CVPR about the network we presented. Firstly, What is the relationship between our proposed layer and the deconvolution layer? And secondly, why are convolutions in low-resolution (LR) space a better choice? These are key questions we tried to answer in the paper, but we were not able to go into as much depth and clarity as we would have liked in the space allowance. To better answer these questions in this note, we first discuss the relationships between the deconvolution layer in the forms of the transposed convolution layer, the sub-pixel convolutional layer and our efficient sub-pixel convolutional layer. We will refer to our efficient sub-pixel convolutional layer as a convolutional layer in LR space to distinguish it from the common sub-pixel convolutional layer. We will then show that for a fixed computational budget and complexity, a network with convolutions exclusively in LR space has more representation power at the same speed than a network that first upsamples the input in high resolution space.

This is a note to share some additional insights for our the CVPR paper

References in corpus (4)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• A guide to convolution arithmetic for deep learning
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Deeply-Recursive Convolutional Network for Image Super-Resolution

Cited by in corpus (26)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• Adversarially Learned Inference
• Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
• Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize
• Super-Resolution via Deep Learning
• Review: Deep Learning in Electron Microscopy
• A Deep Journey into Super-resolution: A survey
• Deep image prior for undersampling high-speed photoacoustic microscopy
• Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances
• Fourier Spectrum Discrepancies in Deep Network Generated Images
• Lightweight Modules for Efficient Deep Learning based Image Restoration
• Brain MRI super-resolution using 3D generative adversarial networks
• Prior-Knowledge and Attention-based Meta-Learning for Few-Shot Learning
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression
• $\Sigma$-net: Systematic Evaluation of Iterative Deep Neural Networks for Fast Parallel MR Image Reconstruction
• Mel-spectrogram augmentation for sequence to sequence voice conversion
• Accelerating Neutron Scattering Data Collection and Experiments Using AI Deep Super-Resolution Learning
• What Information Does a ResNet Compress?
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• On the Robustness and Generalization of Deep Learning Driven Full Waveform Inversion
• Efficient Super Resolution For Large-Scale Images Using Attentional GAN
• Tensor-Generative Adversarial Network with Two-dimensional Sparse Coding: Application to Real-time Indoor Localization
• Deep GAN-Based Cross-Spectral Cross-Resolution Iris Recognition
• A new smart-cropping pipeline for prostate segmentation using deep learning networks
• Finding Correspondences for Optical Flow and Disparity Estimations using a Sub-pixel Convolution-based Encoder-Decoder Network
• Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network