SwapText: Image Based Texts Transfer in Scenes
arXiv:2003.08152
Abstract
Swapping text in scene images while preserving original fonts, colors, sizes and background textures is a challenging task due to the complex interplay between different factors. In this work, we present SwapText, a three-stage framework to transfer texts across scene images. First, a novel text swapping network is proposed to replace text labels only in the foreground image. Second, a background completion network is learned to reconstruct background images. Finally, the generated foreground image and background image are used to generate the word image by the fusion network. Using the proposing framework, we can manipulate the texts of the input images even with severe geometric distortion. Qualitative and quantitative results are presented on several scene text datasets, including regular and irregular text datasets. We conducted extensive experiments to prove the usefulness of our method such as image based text translation, text image synthesis, etc.
arXiv commentAccepted by CVPR 2020
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Conditional Generative Adversarial Nets
- Self-Attention Generative Adversarial Networks
- Generative Adversarial Text to Image Synthesis
- Pixel Recurrent Neural Networks
- Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
- BEGAN: Boundary Equilibrium Generative Adversarial Networks
- Image Inpainting via Generative Multi-column Convolutional Neural Networks
- Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
- TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes
- Controllable Artistic Text Style Transfer via Shape-Matching GAN
- Spatial Fusion GAN for Image Synthesis