An Enhanced Knowledge Injection Model for Commonsense Generation
arXiv:2012.00366
Abstract
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts. Digging the relationship of concepts from scratch is non-trivial, therefore, we retrieve prototypes from external knowledge to assist the understanding of the scenario for better description generation. We integrate two additional modules, namely position indicator and scaling module, into the pretrained encoder-decoder model for prototype modeling to enhance the knowledge injection procedure. We conduct experiment on CommonGen benchmark, and experimental results show that our method significantly improves the performance on all the metrics.
Accepted to COLING 2020
References in corpus (5)
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
- An Information Retrieval Approach to Short Text Conversation
- Two are Better than One: An Ensemble of Retrieval- and Generation-Based Dialog Systems
- HellaSwag: Can a Machine Really Finish Your Sentence?
- Text Generation with Exemplar-based Adaptive Decoding