Evaluating for Diversity in Question Generation over Text
arXiv:2008.07291
Abstract
Generating diverse and relevant questions over text is a task with widespread applications. We argue that commonly-used evaluation metrics such as BLEU and METEOR are not suitable for this task due to the inherent diversity of reference questions, and propose a scheme for extending conventional metrics to reflect diversity. We furthermore propose a variational encoder-decoder model for this task. We show through automatic and human evaluation that our variational model improves diversity without loss of quality, and demonstrate how our evaluation scheme reflects this improvement.
References in corpus (7)
- Conditional Generative Adversarial Nets
- Detecting the overlapping and hierarchical community structure of complex networks
- Finding local community structure in networks
- Variational Autoencoder for Deep Learning of Images, Labels and Captions
- Ladder Variational Autoencoders
- UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
- Semi-Supervised QA with Generative Domain-Adaptive Nets