Stereoscopic Neural Style Transfer

Abstract

This paper presents the first attempt at stereoscopic neural style transfer, which responds to the emerging demand for 3D movies or AR/VR. We start with a careful examination of applying existing monocular style transfer methods to left and right views of stereoscopic images separately. This reveals that the original disparity consistency cannot be well preserved in the final stylization results, which causes 3D fatigue to the viewers. To address this issue, we incorporate a new disparity loss into the widely adopted style loss function by enforcing the bidirectional disparity constraint in non-occluded regions. For a practical real-time solution, we propose the first feed-forward network by jointly training a stylization sub-network and a disparity sub-network, and integrate them in a feature level middle domain. Our disparity sub-network is also the first end-to-end network for simultaneous bidirectional disparity and occlusion mask estimation. Finally, our network is effectively extended to stereoscopic videos, by considering both temporal coherence and disparity consistency. We will show that the proposed method clearly outperforms the baseline algorithms both quantitatively and qualitatively.

Accepted by CVPR2018

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A Learned Representation For Artistic Style
• StyleBank: An Explicit Representation for Neural Image Style Transfer
• Coherent Online Video Style Transfer
• Towards Open-Set Identity Preserving Face Synthesis
• Characterizing and Improving Stability in Neural Style Transfer
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• Learning Selfie-Friendly Abstraction from Artistic Style Images

Cited by in corpus (10)

• Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
• Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• Language-Driven Image Style Transfer
• DAVANet: Stereo Deblurring with View Aggregation
• Neural Abstract Style Transfer for Chinese Traditional Painting
• A Novel Monocular Disparity Estimation Network with Domain Transformation and Ambiguity Learning
• Learning Selfie-Friendly Abstraction from Artistic Style Images
• Image Smoothing via Unsupervised Learning
• Stereo Waterdrop Removal with Row-wise Dilated Attention