Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning

Abstract

Extracting effective deep features to represent content and style information is the key to universal style transfer. Most existing algorithms use VGG19 as the feature extractor, which incurs a high computational cost and impedes real-time style transfer on high-resolution images. In this work, we propose a lightweight alternative architecture - ArtNet, which is based on GoogLeNet, and later pruned by a novel channel pruning method named Zero-channel Pruning specially designed for style transfer approaches. Besides, we propose a theoretically sound sandwich swap transform (S2) module to transfer deep features, which can create a pleasing holistic appearance and good local textures with an improved content preservation ability. By using ArtNet and S2, our method is 2.3 to 107.4 times faster than state-of-the-art approaches. The comprehensive experiments demonstrate that ArtNet can achieve universal, real-time, and high-quality style transfer on high-resolution images simultaneously, (68.03 FPS on 512 times 512 images).

References in corpus (15)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Wide Residual Networks
• One weird trick for parallelizing convolutional neural networks
• A Learned Representation For Artistic Style
• Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Fast Patch-based Style Transfer of Arbitrary Style
• Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
• Preserving Color in Neural Artistic Style Transfer
• Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
• Multi-style Generative Network for Real-time Transfer
• Learning Linear Transformations for Fast Arbitrary Style Transfer
• Demystifying Neural Style Transfer
• Neural Stereoscopic Image Style Transfer

Cited by in corpus (2)

• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets