Style Mixer: Semantic-aware Multi-Style Transfer Network

Abstract

Recent neural style transfer frameworks have obtained astonishing visual quality and flexibility in Single-style Transfer (SST), but little attention has been paid to Multi-style Transfer (MST) which refers to simultaneously transferring multiple styles to the same image. Compared to SST, MST has the potential to create more diverse and visually pleasing stylization results. In this paper, we propose the first MST framework to automatically incorporate multiple styles into one result based on regional semantics. We first improve the existing SST backbone network by introducing a novel multi-level feature fusion module and a patch attention module to achieve better semantic correspondences and preserve richer style details. For MST, we designed a conceptually simple yet effective region-based style fusion module to insert into the backbone. It assigns corresponding styles to content regions based on semantic matching, and then seamlessly combines multiple styles together. Comprehensive evaluations demonstrate that our framework outperforms existing works of SST and MST.

Pacific Graphics 2019

References in corpus (16)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Self-Attention Generative Adversarial Networks
• Squeeze-and-Excitation Networks
• DRAW: A Recurrent Neural Network For Image Generation
• A Learned Representation For Artistic Style
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• Texture Synthesis Using Convolutional Neural Networks
• Universal Style Transfer via Feature Transforms
• Fast Patch-based Style Transfer of Arbitrary Style
• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• Learning Linear Transformations for Fast Arbitrary Style Transfer
• Demystifying Neural Style Transfer
• A Decomposable Attention Model for Natural Language Inference
• Attention-aware Multi-stroke Style Transfer
• MetaStyle: Three-Way Trade-Off Among Speed, Flexibility, and Quality in Neural Style Transfer