Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search

Abstract

We present Grid Beam Search (GBS), an algorithm which extends beam search to allow the inclusion of pre-specified lexical constraints. The algorithm can be used with any model that generates a sequence $\hat{\mathbf{y}}=\{y_0\dots y_T\}$, by maximizing $p(\mathbf{y}|\mathbf{x})=\prod _{t}p(y_t|\mathbf{x};\{y_0\dots y_{t-1}\})$. Lexical constraints take the form of phrases or words that must be present in the output sequence. This is a very general way to incorporate additional knowledge into a model's output without requiring any modification of the model parameters or training data. We demonstrate the feasibility and flexibility of Lexically Constrained Decoding by conducting experiments on Neural Interactive-Predictive Translation, as well as Domain Adaptation for Neural Machine Translation. Experiments show that GBS can provide large improvements in translation quality in interactive scenarios, and that, even without any user input, GBS can be used to achieve significant gains in performance in domain adaptation scenarios.

Accepted as a long paper at ACL 2017

Cited by in corpus (12)

• Survey of Hallucination in Natural Language Generation
• Autoregressive Entity Retrieval
• Machine Translation using Semantic Web Technologies: A Survey
• General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining
• Partially-Supervised Image Captioning
• Unsupervised Editing for Counterfactual Stories
• Reasoning about Actions and State Changes by Injecting Commonsense Knowledge
• Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation
• "Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses
• GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation
• Constraining Linear-chain CRFs to Regular Languages
• Middle-Out Decoding