Fast Patch-based Style Transfer of Arbitrary Style

Abstract

Artistic style transfer is an image synthesis problem where the content of an image is reproduced with the style of another. Recent works show that a visually appealing style transfer can be achieved by using the hidden activations of a pretrained convolutional neural network. However, existing methods either apply (i) an optimization procedure that works for any style image but is very expensive, or (ii) an efficient feedforward network that only allows a limited number of trained styles. In this work we propose a simpler optimization objective based on local matching that combines the content structure and style textures in a single layer of the pretrained network. We show that our objective has desirable properties such as a simpler optimization landscape, intuitive parameter tuning, and consistent frame-by-frame performance on video. Furthermore, we use 80,000 natural images and 80,000 paintings to train an inverse network that approximates the result of the optimization. This results in a procedure for artistic style transfer that is efficient but also allows arbitrary content and style images.

References in corpus (1)

• Very Deep Convolutional Networks for Large-Scale Image Recognition

Cited by in corpus (37)

• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• Quality Evaluation of Arbitrary Style Transfer: Subjective Study and Objective Metric
• AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Meta Networks for Neural Style Transfer
• Dynamic Instance Normalization for Arbitrary Style Transfer
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• Image Super-Resolution by Neural Texture Transfer
• Attention-aware Multi-stroke Style Transfer
• Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer
• Arbitrary Style Transfer via Multi-Adaptation Network
• Wasserstein Style Transfer
• Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology
• Language-Driven Image Style Transfer
• Shape-Texture Debiased Neural Network Training
• Disentangling Structure and Appearance in ViT Feature Space
• ImagineNet: Restyling Apps Using Neural Style Transfer
• Style transfer-based image synthesis as an efficient regularization technique in deep learning
• Fast Universal Style Transfer for Artistic and Photorealistic Rendering
• Style-Aware Normalized Loss for Improving Arbitrary Style Transfer
• Ancient Painting to Natural Image: A New Solution for Painting Processing
• An End-to-end Method for Producing Scanning-robust Stylized QR Codes
• TET-GAN: Text Effects Transfer via Stylization and Destylization
• CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator
• Unpaired Image Translation via Adaptive Convolution-based Normalization
• StyleRemix: An Interpretable Representation for Neural Image Style Transfer
• Semantic Attribute Matching Networks
• Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
• Face Sketch Synthesis with Style Transfer using Pyramid Column Feature
• Global Image Sentiment Transfer
• Image Translation via Fine-grained Knowledge Transfer
• Non-Parametric Neural Style Transfer
• Stay Positive: Non-Negative Image Synthesis for Augmented Reality
• Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art