Semi-Autoregressive Transformer for Image Captioning
arXiv:2106.09436
Abstract
Current state-of-the-art image captioning models adopt autoregressive decoders, \ie they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. To tackle this issue, non-autoregressive image captioning models have recently been proposed to significantly accelerate the speed of inference by generating all words in parallel. However, these non-autoregressive models inevitably suffer from large generation quality degradation since they remove words dependence excessively. To make a better trade-off between speed and quality, we introduce a semi-autoregressive model for image captioning~(dubbed as SATIC), which keeps the autoregressive property in global but generates words parallelly in local . Based on Transformer, there are only a few modifications needed to implement SATIC. Experimental results on the MSCOCO image captioning benchmark show that SATIC can achieve a good trade-off without bells and whistles. Code is available at {\color{magenta}\url{https://github.com/YuanEZhou/satic}}.
arXiv commentRevised
References in corpus (11)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Non-Autoregressive Neural Machine Translation
- Layer Normalization
- A Better Variant of Self-Critical Sequence Training
- Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
- Fast Image Caption Generation with Position Alignment
- Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning
- Semi-Autoregressive Neural Machine Translation
- Non-Autoregressive Coarse-to-Fine Video Captioning