Learning Linear Transformations for Fast Arbitrary Style Transfer

Abstract

Given a random pair of images, an arbitrary style transfer method extracts the feel from the reference image to synthesize an output based on the look of the other content image. Recent arbitrary style transfer methods transfer second order statistics from reference image onto content image via a multiplication between content image features and a transformation matrix, which is computed from features with a pre-determined algorithm. These algorithms either require computationally expensive operations, or fail to model the feature covariance and produce artifacts in synthesized images. Generalized from these methods, in this work, we derive the form of transformation matrix theoretically and present an arbitrary style transfer approach that learns the transformation matrix with a feed-forward network. Our algorithm is highly efficient yet allows a flexible combination of multi-level styles while preserving content affinity during style transfer process. We demonstrate the effectiveness of our approach on four tasks: artistic style transfer, video and photo-realistic style transfer as well as domain adaptation, including comparisons with the state-of-the-art methods.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Learned Representation For Artistic Style
• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• Playing for Data: Ground Truth from Computer Games
• Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
• A Closed-form Solution to Photorealistic Image Stylization
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration

Cited by in corpus (29)

• AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Dynamic Instance Normalization for Arbitrary Style Transfer
• Crossing You in Style: Cross-modal Style Transfer from Music to Visual Arts
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
• Rethinking and Improving the Robustness of Image Style Transfer
• Multi-Frame GAN: Image Enhancement for Stereo Visual Odometry in Low Light
• A Closed-form Solution to Universal Style Transfer
• Diversified Arbitrary Style Transfer via Deep Feature Perturbation
• Multimodal Style Transfer via Graph Cuts
• Language-Driven Image Style Transfer
• Geometric Style Transfer
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• Ultrafast Photorealistic Style Transfer via Neural Architecture Search
• Deep Feature Rotation for Multimodal Image Style Transfer
• ImagineNet: Restyling Apps Using Neural Style Transfer
• Distribution Aligned Multimodal and Multi-Domain Image Stylization
• GLStyleNet: Higher Quality Style Transfer Combining Global and Local Pyramid Features
• WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification
• Domain-Specific Mappings for Generative Adversarial Style Transfer
• StyleRemix: An Interpretable Representation for Neural Image Style Transfer
• Non-Local Representation based Mutual Affine-Transfer Network for Photorealistic Stylization
• What data do we need for training an AV motion planner?
• Learning to Stylize Novel Views
• Separating Content and Style for Unsupervised Image-to-Image Translation
• Non-Parametric Neural Style Transfer
• Multiple Style Transfer via Variational AutoEncoder
• CAMS: Color-Aware Multi-Style Transfer