Fast Patch-based Style Transfer of Arbitrary Style

Abstract

Artistic style transfer is an image synthesis problem where the content of an image is reproduced with the style of another. Recent works show that a visually appealing style transfer can be achieved by using the hidden activations of a pretrained convolutional neural network. However, existing methods either apply (i) an optimization procedure that works for any style image but is very expensive, or (ii) an efficient feedforward network that only allows a limited number of trained styles. In this work we propose a simpler optimization objective based on local matching that combines the content structure and style textures in a single layer of the pretrained network. We show that our objective has desirable properties such as a simpler optimization landscape, intuitive parameter tuning, and consistent frame-by-frame performance on video. Furthermore, we use 80,000 natural images and 80,000 paintings to train an inverse network that approximates the result of the optimization. This results in a procedure for artistic style transfer that is efficient but also allows arbitrary content and style images.

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images

Cited by in corpus (73)

• Multimodal Unsupervised Image-to-Image Translation
• Generative Image Inpainting with Contextual Attention
• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• Style-Transfer via Texture-Synthesis
• Generating Handwritten Chinese Characters using CycleGAN
• Multi-style Generative Network for Real-time Transfer
• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• Neural Style Transfer: A Review
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• Shift-Net: Image Inpainting via Deep Feature Rearrangement
• Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation
• Quality Evaluation of Arbitrary Style Transfer: Subjective Study and Objective Metric
• AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• Similarity-preserving Image-image Domain Adaptation for Person Re-identification
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Meta Networks for Neural Style Transfer
• Fake Generated Painting Detection via Frequency Analysis
• Characterizing and Improving Stability in Neural Style Transfer
• Dynamic Instance Normalization for Arbitrary Style Transfer
• Separating Style and Content for Generalized Style Transfer
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
• Image Super-Resolution by Neural Texture Transfer
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• Attention-aware Multi-stroke Style Transfer
• A Style-Aware Content Loss for Real-time HD Style Transfer
• Face-to-Parameter Translation for Game Character Auto-Creation
• Quantitative Evaluation of Style Transfer
• A Closed-form Solution to Universal Style Transfer
• Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer
• Arbitrary Style Transfer via Multi-Adaptation Network
• Diversified Arbitrary Style Transfer via Deep Feature Perturbation
• Multimodal Style Transfer via Graph Cuts
• Learning from Multi-domain Artistic Images for Arbitrary Style Transfer
• Contextual-based Image Inpainting: Infer, Match, and Translate
• Wasserstein Style Transfer
• Symmetric-Constrained Irregular Structure Inpainting for Brain MRI Registration with Tumor Pathology
• Language-Driven Image Style Transfer
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• Reference-Conditioned Super-Resolution by Neural Texture Transfer
• A Unified Framework for Generalizable Style Transfer: Style and Content Separation
• Parameter-Free Style Projection for Arbitrary Style Transfer
• Disentangling Structure and Appearance in ViT Feature Space
• Shape-Texture Debiased Neural Network Training
• Style transfer-based image synthesis as an efficient regularization technique in deep learning
• Fast Universal Style Transfer for Artistic and Photorealistic Rendering
• Style-Aware Normalized Loss for Improving Arbitrary Style Transfer
• ImagineNet: Restyling Apps Using Neural Style Transfer
• EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
• MVStylizer: An Efficient Edge-Assisted Video Photorealistic Style Transfer System for Mobile Phones
• An End-to-end Method for Producing Scanning-robust Stylized QR Codes
• Ancient Painting to Natural Image: A New Solution for Painting Processing
• Semantic Attribute Matching Networks
• HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions
• Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
• A Pragmatic AI Approach to Creating Artistic Visual Variations by Neural Style Transfer
• CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator
• Unpaired Image Translation via Adaptive Convolution-based Normalization
• TET-GAN: Text Effects Transfer via Stylization and Destylization
• StyleRemix: An Interpretable Representation for Neural Image Style Transfer
• Non-Local Representation based Mutual Affine-Transfer Network for Photorealistic Stylization
• MetaStyle: Three-Way Trade-Off Among Speed, Flexibility, and Quality in Neural Style Transfer
• Non-Parametric Neural Style Transfer
• Image Translation via Fine-grained Knowledge Transfer
• Face Sketch Synthesis with Style Transfer using Pyramid Column Feature
• Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art
• A Content Transformation Block For Image Style Transfer
• Global Image Sentiment Transfer
• Neural Comic Style Transfer: Case Study
• Stay Positive: Non-Negative Image Synthesis for Augmented Reality
• Style and Content Disentanglement in Generative Adversarial Networks
• TE141K: Artistic Text Benchmark for Text Effect Transfer