Texture Networks: Feed-forward Synthesis of Textures and Stylized Images

Abstract

Gatys et al. recently demonstrated that deep networks can generate beautiful textures and stylized images from a single texture example. However, their methods requires a slow and memory-consuming optimization process. We propose here an alternative approach that moves the computational burden to a learning stage. Given a single example of a texture, our approach trains compact feed-forward convolutional networks to generate multiple samples of the same texture of arbitrary size and to transfer artistic style from a given image to any other image. The resulting networks are remarkably light-weight and can generate textures of quality comparable to Gatys~et~al., but hundreds of times faster. More generally, our approach highlights the power and flexibility of generative feed-forward models trained with complex and expressive loss functions.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Texture Synthesis Using Convolutional Neural Networks
• Generative Moment Matching Networks
• A Kernel Method for the Two-Sample Problem
• Training generative neural networks via Maximum Mean Discrepancy optimization

Cited by in corpus (172)

• Instance Normalization: The Missing Ingredient for Fast Stylization
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• VisDA: The Visual Domain Adaptation Challenge
• Unsupervised Cross-Domain Image Generation
• Generative Adversarial Network for Medical Images (MI-GAN)
• Adversarial Transformation Networks: Learning to Generate Adversarial Examples
• Fast Patch-based Style Transfer of Arbitrary Style
• Non-Stationary Texture Synthesis by Adversarial Expansion
• Style-Transfer via Texture-Synthesis
• Swapping Autoencoder for Deep Image Manipulation
• Artistic style transfer for videos and spherical images
• Recent Advances in Deep Learning: An Overview
• Multi-style Generative Network for Real-time Transfer
• Texture Synthesis with Spatial Generative Adversarial Networks
• Neural Style Transfer: A Review
• Learning Linear Transformations for Fast Arbitrary Style Transfer
• Demystifying Neural Style Transfer
• Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption
• Robust Differentiable SVD
• Improving Variational Autoencoder with Deep Feature Consistent and Generative Adversarial Training
• Domain Adaptation for Object Detection via Style Consistency
• Transport-Based Neural Style Transfer for Smoke Simulations
• One-Sided Unsupervised Domain Mapping
• Style Aggregated Network for Facial Landmark Detection
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• A Closed-form Solution to Photorealistic Image Stylization
• Facial-Sketch Synthesis: A New Challenge
• Shift-Net: Image Inpainting via Deep Feature Rearrangement
• Stacked Generative Adversarial Networks
• CalliGAN: Style and Structure-aware Chinese Calligraphy Character Generator
• An Inverse Procedural Modeling Pipeline for SVBRDF Maps
• Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
• Video Frame Interpolation via Adaptive Convolution
• ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks
• 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks
• Landmark Assisted CycleGAN for Cartoon Face Generation
• ZM-Net: Real-time Zero-shot Image Manipulation Network
• High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis
• Exploring the Neural Algorithm of Artistic Style
• HINet: Half Instance Normalization Network for Image Restoration
• AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
• Diversified Texture Synthesis with Feed-forward Networks
• On Demand Solid Texture Synthesis Using Deep 3D Networks
• TextureGAN: Controlling Deep Image Synthesis with Texture Patches
• Incorporating long-range consistency in CNN-based texture generation
• Style Mixer: Semantic-aware Multi-Style Transfer Network
• Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models
• Learning Texture Manifolds with the Periodic Spatial GAN
• Similarity-preserving Image-image Domain Adaptation for Person Re-identification
• Attacks on State-of-the-Art Face Recognition using Attentional Adversarial Attack Generative Network
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• Meta Networks for Neural Style Transfer
• Applying Visual Domain Style Transfer and Texture Synthesis Techniques to Audio - Insights and Challenges
• Synthesizing Filamentary Structured Images with GANs
• Identity-preserving Face Recovery from Stylized Portraits
• Fake Generated Painting Detection via Frequency Analysis
• Take it in your stride: Do we need striding in CNNs?
• A Generative Model for Texture Synthesis based on Optimal Transport between Feature Distributions
• Dynamic Instance Normalization for Arbitrary Style Transfer
• Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
• Characterizing and Improving Stability in Neural Style Transfer
• Separating Style and Content for Generalized Style Transfer
• MW-GAN: Multi-Warping GAN for Caricature Generation with Multi-Style Geometric Exaggeration
• Towards Metamerism via Foveated Style Transfer
• TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer
• Textured Neural Avatars
• Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields
• Few-shot Image Generation via Cross-domain Correspondence
• Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
• GramGAN: Deep 3D Texture Synthesis From 2D Exemplars
• RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
• Deep-Energy: Unsupervised Training of Deep Neural Networks
• Anisotropic Stroke Control for Multiple Artists Style Transfer
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Attention-aware Multi-stroke Style Transfer
• Region-aware Adaptive Instance Normalization for Image Harmonization
• Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition
• DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
• High-Resolution Network for Photorealistic Style Transfer
• Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer
• Rethinking and Improving the Robustness of Image Style Transfer
• Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture
• Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer
• Arbitrary Style Transfer via Multi-Adaptation Network
• Diversified Arbitrary Style Transfer via Deep Feature Perturbation
• Wasserstein Style Transfer
• Learning from Multi-domain Artistic Images for Arbitrary Style Transfer
• Texture Interpolation for Probing Visual Perception
• Mocycle-GAN: Unpaired Video-to-Video Translation
• Language-Driven Image Style Transfer
• Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence
• Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• SurReal: enhancing Surgical simulation Realism using style transfer
• Discovering Fair Representations in the Data Domain
• Domain Adaptation Using Adversarial Learning for Autonomous Navigation
• Fast Face-swap Using Convolutional Neural Networks
• Context-Aware Semantic Inpainting
• Texture Generation with Neural Cellular Automata
• A Unified Framework for Generalizable Style Transfer: Style and Content Separation
• Transposer: Universal Texture Synthesis Using Feature Maps as Transposed Convolution Filter
• Ultrafast Photorealistic Style Transfer via Neural Architecture Search
• Learning to Synthesize Fashion Textures
• Parameter-Free Style Projection for Arbitrary Style Transfer
• Block Shuffle: A Method for High-resolution Fast Style Transfer with Limited Memory
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows
• Synthesizing Diverse, High-Quality Audio Textures
• Deep Network Interpolation for Continuous Imagery Effect Transition
• EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
• Style-Aware Normalized Loss for Improving Arbitrary Style Transfer
• Computing the Spatial Probability of Inclusion inside Partial Contours for Computer Vision Applications
• Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis
• Live Face De-Identification in Video
• Image Recognition Using Scale Recurrent Neural Networks
• Fast Universal Style Transfer for Artistic and Photorealistic Rendering
• Self-Contained Stylization via Steganography for Reverse and Serial Style Transfer
• MVStylizer: An Efficient Edge-Assisted Video Photorealistic Style Transfer System for Mobile Phones
• Interactive Neural Style Transfer with Artists
• GLStyleNet: Higher Quality Style Transfer Combining Global and Local Pyramid Features
• Im2Pencil: Controllable Pencil Illustration from Photographs
• Neural Style Transfer for Point Clouds
• MIXGAN: Learning Concepts from Different Domains for Mixture Generation
• Introspective Generative Modeling: Decide Discriminatively
• Auto-Embedding Generative Adversarial Networks for High Resolution Image Synthesis
• Style is a Distribution of Features
• TET-GAN: Text Effects Transfer via Stylization and Destylization
• Semantic Attribute Matching Networks
• A geometrically aware auto-encoder for multi-texture synthesis
• SAFIN: Arbitrary Style Transfer With Self-Attentive Factorized Instance Normalization
• 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image
• Line Artist: A Multiple Style Sketch to Painting Synthesis Scheme
• Bringing Impressionism to Life with Neural Style Transfer in Come Swim
• Intensity augmentation for domain transfer of whole breast segmentation in MRI
• Awesome Typography: Statistics-Based Text Effects Transfer
• Frame Difference-Based Temporal Loss for Video Stylization
• Towards Learning a Self-inverse Network for Bidirectional Image-to-image Translation
• Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation
• StyleRemix: An Interpretable Representation for Neural Image Style Transfer
• Texture Synthesis with Recurrent Variational Auto-Encoder
• Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
• Interpreting Spatially Infinite Generative Models
• Non-Local Representation based Mutual Affine-Transfer Network for Photorealistic Stylization
• MetaStyle: Three-Way Trade-Off Among Speed, Flexibility, and Quality in Neural Style Transfer
• Every Filter Extracts A Specific Texture In Convolutional Neural Networks
• The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution
• Texture Mixing by Interpolating Deep Statistics via Gaussian Models
• Generative Creativity: Adversarial Learning for Bionic Design
• Tell-the-difference: Fine-grained Visual Descriptor via a Discriminating Referee
• Wasserstein Introspective Neural Networks
• Recognizing Instagram Filtered Images with Feature De-stylization
• Learning to adapt class-specific features across domains for semantic segmentation
• Learning to Stylize Novel Views
• Perceptual Gradient Networks
• Contrastive Feature Loss for Image Prediction
• GPU-Accelerated Mobile Multi-view Style Transfer
• Image Translation via Fine-grained Knowledge Transfer
• Learning to Transfer Visual Effects from Videos to Images
• Restyling Images with the Bangladeshi Paintings Using Neural Style Transfer: A Comprehensive Experiment, Evaluation, and Human Perspective
• Unified Style Transfer
• Multiple Style Transfer via Variational AutoEncoder
• Non-Parametric Neural Style Transfer
• Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network
• Texture synthesis via projection onto multiscale, multilayer statistics
• TE141K: Artistic Text Benchmark for Text Effect Transfer
• Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art
• Coarse-to-Fine Structure-Aware Artistic Style Transfer
• A Content Transformation Block For Image Style Transfer
• Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
• Photo style transfer with consistency losses
• Disentangled Unsupervised Image Translation via Restricted Information Flow
• Unbiased Image Style Transfer