Universal Style Transfer via Feature Transforms

Abstract

Universal style transfer aims to transfer arbitrary visual styles to content images. Existing feed-forward based methods, while enjoying the inference efficiency, are mainly limited by inability of generalizing to unseen styles or compromised visual quality. In this paper, we present a simple yet effective method that tackles these limitations without training on any pre-defined styles. The key ingredient of our method is a pair of feature transforms, whitening and coloring, that are embedded to an image reconstruction network. The whitening and coloring transforms reflect a direct matching of feature covariance of the content image to a given style image, which shares similar spirits with the optimization of Gram matrix based cost in neural style transfer. We demonstrate the effectiveness of our algorithm by generating high-quality stylized images with comparisons to a number of recent methods. We also analyze our method by visualizing the whitened features and synthesizing textures via simple feature coloring.

arXiv commentAccepted by NIPS 2017

Cited by in corpus (21)

• Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
• Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art
• Instance-Aware Domain Generalization for Face Anti-Spoofing
• Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning
• Motion Puzzle: Arbitrary Motion Style Transfer by Body Part
• Robust Differentiable SVD
• Facial-Sketch Synthesis: A New Challenge
• Generation of microbial colonies dataset with deep learning style transfer
• On Demand Solid Texture Synthesis Using Deep 3D Networks
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer
• Structural-analogy from a Single Image Pair
• Painterly Image Harmonization using Diffusion Model
• Crossing You in Style: Cross-modal Style Transfer from Music to Visual Arts
• Deep Learning for Breast MRI Style Transfer with Limited Training Data
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• A proposal and evaluation of new timbre visualisation methods for audio sample browsers
• Deep Feature Rotation for Multimodal Image Style Transfer
• MVStylizer: An Efficient Edge-Assisted Video Photorealistic Style Transfer System for Mobile Phones
• Intrinsic Image Transfer for Illumination Manipulation
• Domain-Specific Mappings for Generative Adversarial Style Transfer