Visual Attribute Transfer through Deep Image Analogy

Abstract

We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene. Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique Deep Image Analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style/texture transfer, color/style swap, sketch/painting to photo, and time lapse.

Accepted by SIGGRAPH 2017

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Texture Synthesis Using Convolutional Neural Networks
• Style-Transfer via Texture-Synthesis
• Preserving Color in Neural Artistic Style Transfer
• Deep Photo Style Transfer
• StyleBank: An Explicit Representation for Neural Image Style Transfer
• Texture Synthesis Using Shallow Convolutional Networks with Random Filters
• High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
• Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

Cited by in corpus (26)

• A Closed-form Solution to Photorealistic Image Stylization
• Cross-domain Correspondence Learning for Exemplar-based Image Translation
• Twin-GAN -- Unpaired Cross-Domain Image Translation with Weight-Sharing GANs
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• A Style-Aware Content Loss for Real-time HD Style Transfer
• ReCoNet: Real-time Coherent Video Style Transfer Network
• A Closed-form Solution to Universal Style Transfer
• Deep Exemplar-based Video Colorization
• High-Resolution Network for Photorealistic Style Transfer
• Mask-Guided Portrait Editing with Conditional GANs
• Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera using Deep Residual Networks
• Bi-level Feature Alignment for Versatile Image Translation and Manipulation
• Ultrafast Photorealistic Style Transfer via Neural Architecture Search
• Fast Universal Style Transfer for Artistic and Photorealistic Rendering
• Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision
• MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval
• StyleNAS: An Empirical Study of Neural Architecture Search to Uncover Surprisingly Fast End-to-End Universal Style Transfer Networks
• Example-Guided Scene Image Synthesis using Masked Spatial-Channel Attention and Patch-Based Self-Supervision
• Cartoon-to-real: An Approach to Translate Cartoon to Realistic Images using GAN
• Non-Local Representation based Mutual Affine-Transfer Network for Photorealistic Stylization
• Human Correspondence Consensus for 3D Object Semantic Understanding
• PortraitGAN for Flexible Portrait Manipulation
• Self-supervised Correlation Mining Network for Person Image Generation
• Global Image Sentiment Transfer
• ViT-Inception-GAN for Image Colourising
• Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X