Learning by Planning: Language-Guided Global Image Editing

Abstract

Recently, language-guided global image editing draws increasing attention with growing application potentials. However, previous GAN-based methods are not only confined to domain-specific, low-resolution data but also lacking in interpretability. To overcome the collective difficulties, we develop a text-to-operation model to map the vague editing language request into a series of editing operations, e.g., change contrast, brightness, and saturation. Each operation is interpretable and differentiable. Furthermore, the only supervision in the task is the target image, which is insufficient for a stable training of sequential decisions. Hence, we propose a novel operation planning algorithm to generate possible editing sequences from the target image as pseudo ground truth. Comparison experiments on the newly collected MA5k-Req dataset and GIER dataset show the advantages of our methods. Code is available at https://jshi31.github.io/T2ONet.

arXiv commentAccepted by CVPR2021

References in corpus (6)

• Sequence to Sequence Learning with Neural Networks
• NIPS 2016 Tutorial: Generative Adversarial Networks
• EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• Bilinear Representation for Language-based Image Editing Using Conditional Generative Adversarial Networks
• Learning to Globally Edit Images with Textual Description