A Learned Representation For Artistic Style

Abstract

The diversity of painting styles represents a rich visual vocabulary for the construction of an image. The degree to which one may learn and parsimoniously capture this visual vocabulary measures our understanding of the higher level features of paintings, if not images in general. In this work we investigate the construction of a single, scalable deep network that can parsimoniously capture the artistic style of a diversity of paintings. We demonstrate that such a network generalizes across a diversity of artistic styles by reducing a painting to a point in an embedding space. Importantly, this model permits a user to explore new painting styles by arbitrarily combining the styles learned from individual paintings. We hope that this work provides a useful step towards building rich models of paintings and offers a window on to the structure of the learned representation of artistic style.

9 pages. 15 pages of Appendix, International Conference on Learning Representations (ICLR) 2017

Cited by in corpus (243)

• Generative Modeling by Estimating Gradients of the Data Distribution
• Image Quality Assessment: Unifying Structure and Texture Similarity
• A Comprehensive Survey of Image Augmentation Techniques for Deep Learning
• U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
• Multimodal Unsupervised Image-to-Image Translation
• Image Super-Resolution Using Very Deep Residual Channel Attention Networks
• Guided Image Generation with Conditional Invertible Neural Networks
• Robust Motion In-betweening
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
• Adversarial Text-to-Image Synthesis: A Review
• TADAM: Task dependent adaptive metric for improved few-shot learning
• Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer
• Adversarial Video Generation on Complex Datasets
• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
• Context Encoding for Semantic Segmentation
• Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Multi-style Generative Network for Real-time Transfer
• High Fidelity Speech Synthesis with Adversarial Networks
• Neural Style Transfer: A Review
• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• ContraGAN: Contrastive Learning for Conditional Image Generation
• Learning Linear Transformations for Fast Arbitrary Style Transfer
• High-Fidelity Image Generation With Fewer Labels
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift
• PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
• Microstructure synthesis using style-based generative adversarial network
• Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study
• Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
• On Self Modulation for Generative Adversarial Networks
• Hybrid Residual Attention Network for Single Image Super Resolution
• Memory Replay GANs: learning to generate images from new categories without forgetting
• Unsupervised Object Segmentation by Redrawing
• Domain Adaptation for Object Detection via Style Consistency
• Style Aggregated Network for Facial Landmark Detection
• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• A Closed-form Solution to Photorealistic Image Stylization
• Facial-Sketch Synthesis: A New Challenge
• Learning Generalisable Omni-Scale Representations for Person Re-Identification
• Shift-Net: Image Inpainting via Deep Feature Rearrangement
• Learning the Effect of Registration Hyperparameters with HyperMorph
• UPHDR-GAN: Generative Adversarial Network for High Dynamic Range Imaging with Unpaired Data
• Domain Generalization with MixStyle
• Efficient Video Object Segmentation via Network Modulation
• A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition
• SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis
• End-to-End Adversarial Text-to-Speech
• Spatially Controllable Image Synthesis with Internal Representation Collaging
• PConv: Simple yet Effective Convolutional Layer for Generative Adversarial Network
• Improving the Transferability of Adversarial Examples with Arbitrary Style Transfer
• Simple yet Effective Way for Improving the Performance of GAN
• Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation
• NeRF-VAE: A Geometry Aware 3D Scene Generative Model
• Learning to See: You Are What You See
• Metalearned Neural Memory
• AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
• Data-driven Crop Growth Simulation on Time-varying Generated Images using Multi-conditional Generative Adversarial Networks
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation
• Learning Texture Manifolds with the Periodic Spatial GAN
• Controllable Person Image Synthesis with Attribute-Decomposed GAN
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• Towards Feature Space Adversarial Attack
• Daft-Exprt: Cross-Speaker Prosody Transfer on Any Text for Expressive Speech Synthesis
• Controllable Video Captioning with an Exemplar Sentence
• Reinforcing Generated Images via Meta-learning for One-Shot Fine-Grained Visual Recognition
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Transferring GANs: generating images from limited data
• Adapting Object Detectors with Conditional Domain Normalization
• ConvS2S-VC: Fully convolutional sequence-to-sequence voice conversion
• Applying Visual Domain Style Transfer and Texture Synthesis Techniques to Audio - Insights and Challenges
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis
• Characterizing and Improving Stability in Neural Style Transfer
• Adjustable Real-time Style Transfer
• Stereoscopic Neural Style Transfer
• Improving One-stage Visual Grounding by Recursive Sub-query Construction
• StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion
• Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields
• Guided Image-to-Image Translation with Bi-Directional Feature Transformation
• Image Generation From Small Datasets via Batch Statistics Adaptation
• MixStyle Neural Networks for Domain Generalization and Adaptation
• Learning to Segment Every Thing
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• Does enhanced shape bias improve neural network robustness to common corruptions?
• PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
• Style Example-Guided Text Generation using Generative Adversarial Transformers
• Transformation-based Adversarial Video Prediction on Large-Scale Data
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations
• Generating unrepresented proportions of geological facies using Generative Adversarial Networks
• A Style-Aware Content Loss for Real-time HD Style Transfer
• Image Morphing with Perceptual Constraints and STN Alignment
• Lightweight Generative Adversarial Networks for Text-Guided Image Manipulation
• Region-aware Adaptive Instance Normalization for Image Harmonization
• cGANs with Multi-Hinge Loss
• ReCoNet: Real-time Coherent Video Style Transfer Network
• Controlling Perceptual Factors in Neural Style Transfer
• Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer
• High-Resolution Network for Photorealistic Style Transfer
• Combining Local and Global Pose Estimation for Precise Tracking of Similar Objects
• COCO-GAN: Generation by Parts via Conditional Coordinating
• A Closed-form Solution to Universal Style Transfer
• Quantitative Evaluation of Style Transfer
• Generating Electronic Health Records with Multiple Data Types and Constraints
• Generative Feature Replay For Class-Incremental Learning
• Segmenting Objects in Day and Night:Edge-Conditioned CNN for Thermal Image Semantic Segmentation
• Diversified Arbitrary Style Transfer via Deep Feature Perturbation
• Demystifying Inter-Class Disentanglement
• Combining Learned Lyrical Structures and Vocabulary for Improved Lyric Generation
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Positional Normalization
• Benefits of Linear Conditioning with Metadata for Image Segmentation
• Image Synthesis From Reconfigurable Layout and Style
• Meta Feature Modulator for Long-tailed Recognition
• Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence
• Styleformer: Transformer based Generative Adversarial Networks with Style Vector
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization
• MineGAN: effective knowledge transfer from GANs to target domains with few images
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• JGAN: A Joint Formulation of GAN for Synthesizing Images and Labels
• Label-Noise Robust Generative Adversarial Networks
• cGANs with Conditional Convolution Layer
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation
• ManiGAN: Text-Guided Image Manipulation
• Self-labeled Conditional GANs
• StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contexts
• A Unified Framework for Generalizable Style Transfer: Style and Content Separation
• CoCosNet v2: Full-Resolution Correspondence Learning for Image Translation
• Parameter-Free Style Projection for Arbitrary Style Transfer
• Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer
• Meta-Learning Dynamics Forecasting Using Task Inference
• Deep Feature Rotation for Multimodal Image Style Transfer
• Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Telescoping Density-Ratio Estimation
• Multiscale style transfer based on a Laplacian pyramid for traditional Chinese painting
• From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition
• Attention-Guided NIR Image Colorization via Adaptive Fusion of Semantic and Texture Clues
• XCycles Backprojection Acoustic Super-Resolution
• Semantically Multi-modal Image Synthesis
• Reinforcement Learning Enhanced Quantum-inspired Algorithm for Combinatorial Optimization
• Variational Conditional GAN for Fine-grained Controllable Image Generation
• Style-Aware Normalized Loss for Improving Arbitrary Style Transfer
• Deep Network Interpolation for Continuous Imagery Effect Transition
• Taking Control of Intra-class Variation in Conditional GANs Under Weak Supervision
• EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
• Score-Guided Generative Adversarial Networks
• Fast Universal Style Transfer for Artistic and Photorealistic Rendering
• Synthesizing Diverse, High-Quality Audio Textures
• Implicit Autoencoders
• Modulating Image Restoration with Continual Levels via Adaptive Feature Modification Layers
• A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers
• Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation
• Class-Distinct and Class-Mutual Image Generation with GANs
• Self-Contained Stylization via Steganography for Reverse and Serial Style Transfer
• Efficient Learning of Generative Models via Finite-Difference Score Matching
• Enhance Images as You Like with Unpaired Learning
• Co-supervised learning paradigm with conditional generative adversarial networks for sample-efficient classification
• Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise
• Deep Detail Enhancement for Any Garment
• MVStylizer: An Efficient Edge-Assisted Video Photorealistic Style Transfer System for Mobile Phones
• A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
• GAN Memory with No Forgetting
• Global and Local Alignment Networks for Unpaired Image-to-Image Translation
• Dance with Flow: Two-in-One Stream Action Detection
• Automatic segmentation of spinal multiple sclerosis lesions: How to generalize across MRI contrasts?
• Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization
• A Unified Hyper-GAN Model for Unpaired Multi-contrast MR Image Translation
• An Empirical Study of Batch Normalization and Group Normalization in Conditional Computation
• Deep Context-Aware Novelty Detection
• Frame Difference-Based Temporal Loss for Video Stylization
• SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization
• Stable Rank Normalization for Improved Generalization in Neural Networks and GANs
• Fashion Editing with Adversarial Parsing Learning
• Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks
• Unsupervised Multimodal Video-to-Video Translation via Self-Supervised Learning
• Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
• 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
• Unsupervised Stylish Image Description Generation via Domain Layer Norm
• StyleRemix: An Interpretable Representation for Neural Image Style Transfer
• Style is a Distribution of Features
• How good is my GAN?
• Uncorrelated Feature Encoding for Faster Image Style Transfer
• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
• SLGAN: Style- and Latent-guided Generative Adversarial Network for Desirable Makeup Transfer and Removal
• TinyGAN: Distilling BigGAN for Conditional Image Generation
• CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator
• Omni-GAN: On the Secrets of cGANs and Beyond
• Data Augmentation via Mixed Class Interpolation using Cycle-Consistent Generative Adversarial Networks Applied to Cross-Domain Imagery
• Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
• Semantic Text-to-Face GAN -ST^2FG
• Multi-defect microscopy image restoration under limited data conditions
• CIZSL++: Creativity Inspired Generative Zero-Shot Learning
• Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360° Panoramic Imagery
• Multiple Style-Transfer in Real-Time
• Kunster -- AR Art Video Maker -- Real time video neural style transfer on mobile devices
• Image Style Transfer and Content-Style Disentanglement
• CoPE: Conditional image generation using Polynomial Expansions
• Robustness of Facial Recognition to GAN-based Face-morphing Attacks
• The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution
• Recognizing Instagram Filtered Images with Feature De-stylization
• Texture Mixing by Interpolating Deep Statistics via Gaussian Models
• Generative Creativity: Adversarial Learning for Bionic Design
• Learning Portrait Style Representations
• A Method for Arbitrary Instance Style Transfer
• TS-Net: OCR Trained to Switch Between Text Transcription Styles
• Improving the Generalization of Meta-learning on Unseen Domains via Adversarial Shift
• MetaStyle: Three-Way Trade-Off Among Speed, Flexibility, and Quality in Neural Style Transfer
• PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
• A Content Transformation Block For Image Style Transfer
• Towards Low-Resource StarGAN Voice Conversion using Weight Adaptive Instance Normalization
• Conditional Transferring Features: Scaling GANs to Thousands of Classes with 30% Less High-quality Data for Training
• Unaligned Image-to-Sequence Transformation with Loop Consistency
• Double Forward Propagation for Memorized Batch Normalization
• TE141K: Artistic Text Benchmark for Text Effect Transfer
• Efficient Semantic Image Synthesis via Class-Adaptive Normalization
• Style Decomposition for Improved Neural Style Transfer
• Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos
• Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifiers
• Identifying and Exploiting Structures for Reliable Deep Learning
• Stay Positive: Non-Negative Image Synthesis for Augmented Reality
• Learning to Generate Multiple Style Transfer Outputs for an Input Sentence
• Non-Parametric Neural Style Transfer
• GPU-Accelerated Mobile Multi-view Style Transfer
• Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
• Active Authentication using an Autoencoder regularized CNN-based One-Class Classifier
• Image Translation via Fine-grained Knowledge Transfer
• Photo style transfer with consistency losses
• Invertible Tone Mapping with Selectable Styles
• 3D Topology Transformation with Generative Adversarial Networks
• Improving Shape Deformation in Unsupervised Image-to-Image Translation
• Semantic Image Fusion
• Unbiased Image Style Transfer
• Deep Automodulators
• Channel Decomposition into Painting Actions
• Context Normalization Layer with Applications
• Restyling Images with the Bangladeshi Paintings Using Neural Style Transfer: A Comprehensive Experiment, Evaluation, and Human Perspective
• Delving into Rectifiers in Style-Based Image Translation
• Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
• Unified Style Transfer