Self-Supervised Feature Learning by Learning to Spot Artifacts

Abstract

We introduce a novel self-supervised learning method based on adversarial training. Our objective is to train a discriminator network to distinguish real images from images with synthetic artifacts, and then to extract features from its intermediate layers that can be transferred to other data domains and tasks. To generate images with artifacts, we pre-train a high-capacity autoencoder and then we use a damage and repair strategy: First, we freeze the autoencoder and damage the output of the encoder by randomly dropping its entries. Second, we augment the decoder with a repair network, and train it in an adversarial manner against the discriminator. The repair network helps generate more realistic images by inpainting the dropped feature entries. To make the discriminator focus on the artifacts, we also make it predict what entries in the feature were dropped. We demonstrate experimentally that features learned by creating and spotting artifacts achieve state of the art performance in several benchmarks.

arXiv commentCVPR 2018 (spotlight)

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Deep Residual Learning for Image Recognition
• Understanding Neural Networks Through Deep Visualization
• Improved Techniques for Training GANs
• Fully Convolutional Networks for Semantic Segmentation
• Adversarial Feature Learning
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Unsupervised Visual Representation Learning by Context Prediction
• Stacked What-Where Auto-encoders
• Unsupervised Learning of Visual Representations using Videos
• Data-dependent Initializations of Convolutional Neural Networks
• Learning to See by Moving
• Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks
• Convolutional Clustering for Unsupervised Learning
• Colorization as a Proxy Task for Visual Understanding
• Learning Features by Watching Objects Move
• Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

Cited by in corpus (6)

• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• Attribute Restoration Framework for Anomaly Detection
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• Image Enhanced Rotation Prediction for Self-Supervised Learning
• HDRVideo-GAN: Deep Generative HDR Video Reconstruction
• Leveraging Hidden Structure in Self-Supervised Learning