The Image Local Autoregressive Transformer
arXiv:2106.02514
Abstract
Recently, AutoRegressive (AR) models for the whole image generation empowered by transformers have achieved comparable or even better performance to Generative Adversarial Networks (GANs). Unfortunately, directly applying such AR models to edit/change local image regions, may suffer from the problems of missing global information, slow inference speed, and information leakage of local guidance. To address these limitations, we propose a novel model -- image Local Autoregressive Transformer (iLAT), to better facilitate the locally guided image synthesis. Our iLAT learns the novel local discrete representations, by the newly proposed local autoregressive (LA) transformer of the attention mask and convolution mechanism. Thus iLAT can efficiently synthesize the local image regions by key guidance information. Our iLAT is evaluated on various locally guided image syntheses, such as pose-guided person image synthesis and face editing. Both the quantitative and qualitative results show the efficacy of our model.
arXiv commentAccepted by NeurIPS2021
References in corpus (11)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- LSTM: A Search Space Odyssey
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Language Models are Few-Shot Learners
- Neural Discrete Representation Learning
- Conditional Image Generation with PixelCNN Decoders
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Pixel Recurrent Neural Networks
- Zero-Shot Text-to-Image Generation
- Long Text Generation via Adversarial Training with Leaked Information
- Large Scale Image Completion via Co-Modulated Generative Adversarial Networks