Photorealistic Style Transfer via Wavelet Transforms

Abstract

Recent style transfer models have provided promising artistic results. However, given a photograph as a reference style, existing methods are limited by spatial distortions or unrealistic artifacts, which should not happen in real photographs. We introduce a theoretically sound correction to the network architecture that remarkably enhances photorealism and faithfully transfers the style. The key ingredient of our method is wavelet transforms that naturally fits in deep networks. We propose a wavelet corrected transfer based on whitening and coloring transforms (WCT${}^2$) that allows features to preserve their structural information and statistical properties of VGG feature space during stylization. This is the first and the only end-to-end model that can stylize a $1024\times 1024$ resolution image in 4.7 seconds, giving a pleasing and photorealistic quality without any post-processing. Last but not least, our model provides a stable video stylization without temporal constraints. Our code, generated images, and pre-trained models are all available at https://github.com/ClovaAI/WCT2.

arXiv commentAccepted to ICCV 2019, Code and data: https://github.com/ClovaAI/WCT2

References in corpus (4)

• Adam: A Method for Stochastic Optimization
• NSML: Meet the MLaaS platform with a real-world case study
• Wavelet Convolutional Neural Networks for Texture Classification
• A Closed-form Solution to Photorealistic Image Stylization

Cited by in corpus (7)

• Domain Adaptation for Object Detection via Style Consistency
• Wavelet Integrated CNNs for Noise-Robust Image Classification
• WaveSNet: Wavelet Integrated Deep Networks for Image Segmentation
• Remove Appearance Shift for Ultrasound Image Segmentation via Fast and Universal Style Transfer
• Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision
• Example-Guided Scene Image Synthesis using Masked Spatial-Channel Attention and Patch-Based Self-Supervision
• Synthesizing Photorealistic Images with Deep Generative Learning