Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation

Abstract

Natural question generation (QG) aims to generate questions from a passage and an answer. Previous works on QG either (i) ignore the rich structure information hidden in text, (ii) solely rely on cross-entropy loss that leads to issues like exposure bias and inconsistency between train/test measurement, or (iii) fail to fully exploit the answer information. To address these limitations, in this paper, we propose a reinforcement learning (RL) based graph-to-sequence (Graph2Seq) model for QG. Our model consists of a Graph2Seq generator with a novel Bidirectional Gated Graph Neural Network based encoder to embed the passage, and a hybrid evaluator with a mixed objective combining both cross-entropy and RL losses to ensure the generation of syntactically and semantically valid text. We also introduce an effective Deep Alignment Network for incorporating the answer information into the passage at both the word and contextual levels. Our model is end-to-end trainable and achieves new state-of-the-art scores, outperforming existing methods by a significant margin on the standard SQuAD benchmark.

arXiv comment17 pages. Accepted by ICLR 2020. Final version (fix typo in figure)

References in corpus (15)

• Semi-Supervised Classification with Graph Convolutional Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Neural Message Passing for Quantum Chemistry
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• A Deep Reinforced Model for Abstractive Summarization
• Variational Dropout and the Local Reparameterization Trick
• Question Answering and Question Generation as Dual Tasks
• Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks
• Learning to Generate Questions by Learning What not to Generate
• A Unified Query-based Generative Model for Question Generation and Question Answering
• GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
• Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model
• A Syntactic Approach to Domain-Specific Automatic Question Generation
• Automating Reading Comprehension by Generating Question and Answer Pairs
• Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus

Cited by in corpus (16)

• A Survey of Knowledge-Enhanced Text Generation
• Improved Automatic Summarization of Subroutines via Attention to File Context
• Graph Neural Networks for Natural Language Processing: A Survey
• Multi-hop Question Generation with Graph Convolutional Network
• End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems
• Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks
• Sequential Recommendation with Graph Neural Networks
• Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem
• Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning
• Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing
• HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebooks
• Semantic-Based Self-Critical Training For Question Generation
• Stronger Transformers for Neural Multi-Hop Question Generation
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation
• Automatic Distractor Generation for Multiple Choice Questions in Standard Tests
• ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Models