Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

Abstract

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets. In this work, we take a step towards general scene-level image editing by developing an automatic interaction-free object removal model. Our model learns to find and remove objects from general scene images using image-level labels and unpaired data in a generative adversarial network (GAN) framework. We achieve this with two key contributions: a two-stage editor architecture consisting of a mask generator and image in-painter that co-operate to remove objects, and a novel GAN based prior for the mask generator that allows us to flexibly incorporate knowledge about object shapes. We experimentally show on two datasets that our method effectively removes a wide variety of objects using weak supervision only

Cited by in corpus (16)

• Media Forensics and DeepFakes: an overview
• MONet: Unsupervised Scene Decomposition and Representation
• SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
• Privacy Intelligence: A Survey on Image Privacy in Online Social Networks
• Noise or Signal: The Role of Image Backgrounds in Object Recognition
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
• End-to-End Learning of Geometric Deformations of Feature Maps for Virtual Try-On
• DeepUSPS: Deep Robust Unsupervised Saliency Prediction With Self-Supervision
• DriveGuard: Robustification of Automated Driving Systems with Deep Spatio-Temporal Convolutional Autoencoder
• PD-GAN: Probabilistic Diverse GAN for Image Inpainting
• Recurrent Feature Reasoning for Image Inpainting
• Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation
• Semantic Image Manipulation Using Scene Graphs
• Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation
• PanDA: Panoptic Data Augmentation
• Semantic Editing On Segmentation Map Via Multi-Expansion Loss