Semantically Consistent Image Completion with Fine-grained Details

Abstract

Image completion has achieved significant progress due to advances in generative adversarial networks (GANs). Albeit natural-looking, the synthesized contents still lack details, especially for scenes with complex structures or images with large holes. This is because there exists a gap between low-level reconstruction loss and high-level adversarial loss. To address this issue, we introduce a perceptual network to provide mid-level guidance, which measures the semantical similarity between the synthesized and original contents in a similarity-enhanced space. We conduct a detailed analysis on the effects of different losses and different levels of perceptual features in image completion, showing that there exist complementarity between adversarial training and perceptual features. By combining them together, our model can achieve nearly seamless fusion results in an end-to-end manner. Moreover, we design an effective lightweight generator architecture, which can achieve effective image inpainting with far less parameters. Evaluated on CelebA Face and Paris StreetView dataset, our proposed method significantly outperforms existing methods.

9 pages plus 2-page supplement

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Multi-Scale Context Aggregation by Dilated Convolutions
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• Improved Techniques for Training GANs
• Generative Face Completion

Cited by in corpus (5)

• Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails
• Deep Inception Generative Network for Cognitive Image Inpainting
• Perceptually Motivated Method for Image Inpainting Comparison
• Geometry-Aware Face Completion and Editing
• TricycleGAN: Unsupervised Image Synthesis and Segmentation Based on Shape Priors