TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations
arXiv:2103.15982
Abstract
Image inpainting is the task of plausibly restoring missing pixels within a hole region that is to be removed from a target image. Most existing technologies exploit patch similarities within the image, or leverage large-scale training data to fill the hole using learned semantic and texture information. However, due to the ill-posed nature of the inpainting task, such methods struggle to complete larger holes containing complicated scenes. In this paper, we propose TransFill, a multi-homography transformed fusion method to fill the hole by referring to another source image that shares scene contents with the target image. We first align the source image to the target image by estimating multiple homographies guided by different depth levels. We then learn to adjust the color and apply a pixel-level warping to each homography-warped source image to make it more consistent with the target. Finally, a pixel-level fusion module is learned to selectively merge the different proposals. Our method achieves state-of-the-art performance on pairs of images across a variety of wide baselines and color differences, and generalizes to user-provided image pairs.
arXiv commentAccepted by CVPR2021
References in corpus (8)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
- Video Inpainting of Complex Scenes
- Deep Image Homography Estimation
- Improving the Harmony of the Composite Image by Spatial-Separated Attention Module
- SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
- DeepMeshFlow: Content Adaptive Mesh Deformation for Robust Image Registration