Spectrally Consistent UNet for High Fidelity Image Transformations

Abstract

Convolutional Neural Networks (CNNs) are the current de-facto models used for many imaging tasks due to their high learning capacity as well as their architectural qualities. The ubiquitous UNet architecture provides an efficient and multi-scale solution that combines local and global information. Despite the success of UNet architectures, the use of upsampling layers can cause artefacts. In this work, a method for assessing the structural biases of UNets and the effects these have on the outputs is presented, characterising their impact in the Fourier domain. A new upsampling module is proposed, based on a novel use of the Guided Image Filter, that provides spectrally consistent outputs when used in a UNet architecture, forming the Guided UNet (GUNet). The GUNet architecture is applied and evaluated for example applications of inverse tone mapping/dynamic range expansion and colourisation from grey-scale images and is shown to provide higher fidelity outputs.

References in corpus (9)

• Deep Learning in Neural Networks: An Overview
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• HDR image reconstruction from a single exposure using deep CNNs
• Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize
• Deep Chain HDRI: Reconstructing a High Dynamic Range Image from a Single Low Dynamic Range Image
• Hybrid Loss for Learning Single-Image-based HDR Reconstruction
• Learning High Dynamic Range from Outdoor Panoramas
• The Devil is in the Decoder: Classification, Regression and GANs