EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning

Abstract

Over the last few years, deep learning techniques have yielded significant improvements in image inpainting. However, many of these techniques fail to reconstruct reasonable structures as they are commonly over-smoothed and/or blurry. This paper develops a new approach for image inpainting that does a better job of reproducing filled regions exhibiting fine details. We propose a two-stage adversarial model EdgeConnect that comprises of an edge generator followed by an image completion network. The edge generator hallucinates edges of the missing region (both regular and irregular) of the image, and the image completion network fills in the missing regions using hallucinated edges as a priori. We evaluate our model end-to-end over the publicly available datasets CelebA, Places2, and Paris StreetView, and show that it outperforms current state-of-the-art techniques quantitatively and qualitatively. Code and models available at: https://github.com/knazeri/edge-connect

Code and data: https://github.com/knazeri/edge-connect

References in corpus (6)

• Conditional Generative Adversarial Nets
• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• Richer Convolutional Features for Edge Detection
• Are GANs Created Equal? A Large-Scale Study
• Is Generator Conditioning Causally Related to GAN Performance?
• Contextual-based Image Inpainting: Infer, Match, and Translate

Cited by in corpus (91)

• A Survey of Human-in-the-loop for Machine Learning
• PiiGAN: Generative Adversarial Networks for Pluralistic Image Inpainting
• Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
• SESAME: Semantic Editing of Scenes by Adding, Manipulating or Erasing Objects
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Edge and Identity Preserving Network for Face Super-Resolution
• StructureFlow: Image Inpainting via Structure-aware Appearance Flow
• Learnable Gated Temporal Shift Module for Deep Video Inpainting
• MTRNet++: One-stage Mask-based Scene Text Eraser
• Texture Memory-Augmented Deep Patch-Based Image Inpainting
• Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Deep Unfolding with Normalizing Flow Priors for Inverse Problems
• Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations
• Image Inpainting with Learnable Bidirectional Attention Maps
• Deep Long Audio Inpainting
• Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
• Deep Face Video Inpainting via UV Mapping
• LaFIn: Generative Landmark Guided Face Inpainting
• Delving into Crispness: Guided Label Refinement for Crisp Edge Detection
• Learning Joint Spatial-Temporal Transformations for Video Inpainting
• A Deep Learning Framework to Reconstruct Face under Mask
• Diverse Image Inpainting with Bidirectional and Autoregressive Transformers
• Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment
• Common Spatial Generative Adversarial Networks based EEG Data Augmentation for Cross-Subject Brain-Computer Interface
• DeViT: Deformed Vision Transformers in Video Inpainting
• Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
• Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes
• PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain
• Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN
• Controllable and Progressive Image Extrapolation
• High Fidelity Face Manipulation with Extreme Poses and Expressions
• Image Inpainting Guided by Coherence Priors of Semantics and Textures
• WaveFill: A Wavelet-based Generation Network for Image Inpainting
• Learning Disentangled Feature Representation for Hybrid-distorted Image Restoration
• Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
• Privacy Leakage of SIFT Features via Deep Generative Model based Image Reconstruction
• Inpainting hydrodynamical maps with deep learning
• EdgeNet: A novel approach for Arabic numeral classification
• A Benchmark and Baseline for Language-Driven Image Editing
• Multi-Plane Program Induction with 3D Box Priors
• Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis
• Design and Development of a Web-based Tool for Inpainting of Dissected Aortae in Angiography Images
• Image Inpainting with Edge-guided Learnable Bidirectional Attention Maps
• Learning Oracle Attention for High-fidelity Face Completion
• TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations
• Learning to Incorporate Structure Knowledge for Image Inpainting
• Image Restoration from Parametric Transformations using Generative Models
• Context-Aware Image Inpainting with Learned Semantic Priors
• PEEL: A Provable Removal Attack on Deep Hiding
• Bringing Old Photos Back to Life
• Finding and Fixing Spurious Patterns with Explanations
• PD-GAN: Probabilistic Diverse GAN for Image Inpainting
• Future Urban Scenes Generation Through Vehicles Synthesis
• VCNet: A Robust Approach to Blind Image Inpainting
• Deep Fusion Network for Image Completion
• Fully Automated Image De-fencing using Conditional Generative Adversarial Networks
• To Beta or Not To Beta: Information Bottleneck for DigitaL Image Forensics
• Instance Separation Emerges from Inpainting
• Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
• Fashion Editing with Adversarial Parsing Learning
• SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches
• One-Stage Inpainting with Bilateral Attention and Pyramid Filling Block
• Let's See Clearly: Contaminant Artifact Removal for Moving Cameras
• Road images augmentation with synthetic traffic signs using neural networks
• Flow-edge Guided Video Completion
• Free-Form Image Inpainting via Contrastive Attention Network
• FaceEraser: Removing Facial Parts for Augmented Reality
• Large Hole Image Inpainting With Compress-Decompression Network
• Residual Squeeze-and-Excitation Network for Fast Image Deraining
• DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting
• Modeling Artistic Workflows for Image Generation and Editing
• Painting Outside as Inside: Edge Guided Image Outpainting via Bidirectional Rearrangement with Progressive Step Learning
• Deep Likelihood Network for Image Restoration with Multiple Degradation Levels
• Test-Time Adaptation for Out-of-distributed Image Inpainting
• Copy-Move Forgery Classification via Unsupervised Domain Adaptation
• Multimodal Image Outpainting With Regularized Normalized Diversification
• Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation
• Nested Scale Editing for Conditional Image Synthesis
• Pixel-wise Dense Detector for Image Inpainting
• Interactive Image Restoration
• Eliminating Shadow Artifacts via Generative Inpainting Networks to Quantify Vascular Changes of the Choroid
• CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction
• DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Nets
• Syn2Real: Forgery Classification via Unsupervised Domain Adaptation
• 3D Ken Burns Effect from a Single Image
• Learning by Planning: Language-Guided Global Image Editing
• GIID-Net: Generalizable Image Inpainting Detection via Neural Architecture Search and Attention
• Detailed Avatar Recovery from Single Image
• Phenotype-preserving metric design for high-content image reconstruction by generative inpainting