Universal Style Transfer via Feature Transforms

Abstract

Universal style transfer aims to transfer arbitrary visual styles to content images. Existing feed-forward based methods, while enjoying the inference efficiency, are mainly limited by inability of generalizing to unseen styles or compromised visual quality. In this paper, we present a simple yet effective method that tackles these limitations without training on any pre-defined styles. The key ingredient of our method is a pair of feature transforms, whitening and coloring, that are embedded to an image reconstruction network. The whitening and coloring transforms reflect a direct matching of feature covariance of the content image to a given style image, which shares similar spirits with the optimization of Gram matrix based cost in neural style transfer. We demonstrate the effectiveness of our algorithm by generating high-quality stylized images with comparisons to a number of recent methods. We also analyze our method by visualizing the whitened features and synthesizing textures via simple feature coloring.

Accepted by NIPS 2017

Cited by in corpus (21)

• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Instance-Aware Domain Generalization for Face Anti-Spoofing
• Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
• Motion Puzzle: Arbitrary Motion Style Transfer by Body Part
• Robust Differentiable SVD
• Facial-Sketch Synthesis: A New Challenge
• Generation of microbial colonies dataset with deep learning style transfer
• On Demand Solid Texture Synthesis Using Deep 3D Networks
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer
• Structural-analogy from a Single Image Pair
• Painterly Image Harmonization using Diffusion Model
• Crossing You in Style: Cross-modal Style Transfer from Music to Visual Arts
• Deep Learning for Breast MRI Style Transfer with Limited Training Data
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• A proposal and evaluation of new timbre visualisation methods for audio sample browsers
• Deep Feature Rotation for Multimodal Image Style Transfer
• MVStylizer: An Efficient Edge-Assisted Video Photorealistic Style Transfer System for Mobile Phones
• Intrinsic Image Transfer for Illumination Manipulation
• Domain-Specific Mappings for Generative Adversarial Style Transfer