Semi-parametric Image Synthesis

Abstract

We present a semi-parametric approach to photographic image synthesis from semantic layouts. The approach combines the complementary strengths of parametric and nonparametric techniques. The nonparametric component is a memory bank of image segments constructed from a training set of images. Given a novel semantic layout at test time, the memory bank is used to retrieve photographic references that are provided as source material to a deep network. The synthesis is performed by a deep network that draws on the provided photographic material. Experiments on multiple semantic segmentation datasets show that the presented approach yields considerably more realistic images than recent purely parametric techniques. The results are shown in the supplementary video at https://youtu.be/U4Q98lenGLQ

Published at the Conference on Computer Vision and Pattern Recognition (CVPR 2018)

References in corpus (6)

• Generative Adversarial Text to Image Synthesis
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Playing for Data: Ground Truth from Computer Games
• Single-Image Depth Perception in the Wild
• Deep Image Harmonization
• Statistics of Deep Generated Images

Cited by in corpus (6)

• Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
• Rethinking Spatially-Adaptive Normalization
• Semi-parametric Image Inpainting
• Pixel Level Data Augmentation for Semantic Image Segmentation using Generative Adversarial Networks
• Semantic Road Layout Understanding by Generative Adversarial Inpainting
• MOC-GAN: Mixing Objects and Captions to Generate Realistic Images