The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution

Abstract

While implicit generative models such as GANs have shown impressive results in high quality image reconstruction and manipulation using a combination of various losses, we consider a simpler approach leading to surprisingly strong results. We show that texture loss alone allows the generation of perceptually high quality images. We provide a better understanding of texture constraining mechanism and develop a novel semantically guided texture constraining method for further improvement. Using a recently developed perceptual metric employing "deep features" and termed LPIPS, the method obtains state-of-the-art results. Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features. Using texture information, off-the-shelf deep classification networks (without training) perform as well as the best performing (tuned and calibrated) LPIPS metrics. The code is publicly available.

arXiv comment19 pages, 14 figures

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• The Perception-Distortion Tradeoff
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• A Learned Representation For Artistic Style
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• Semantic Understanding of Scenes through the ADE20K Dataset
• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• Demystifying Neural Style Transfer
• StyleBank: An Explicit Representation for Neural Image Style Transfer
• Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
• Improved Style Transfer by Respecting Inter-layer Correlations
• Super-resolution Using Constrained Deep Texture Synthesis