Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System

Abstract

In real-world question-answering (QA) systems, ill-formed questions, such as wrong words, ill word order, and noisy expressions, are common and may prevent the QA systems from understanding and answering them accurately. In order to eliminate the effect of ill-formed questions, we approach the question refinement task and propose a unified model, QREFINE, to refine the ill-formed questions to well-formed question. The basic idea is to learn a Seq2Seq model to generate a new question from the original one. To improve the quality and retrieval performance of the generated questions, we make two major improvements: 1) To better encode the semantics of ill-formed questions, we enrich the representation of questions with character embedding and the recent proposed contextual word embedding such as BERT, besides the traditional context-free word embeddings; 2) To make it capable to generate desired questions, we train the model with deep reinforcement learning techniques that considers an appropriate wording of the generation as an immediate reward and the correlation between generated question and answer as time-delayed long-term rewards. Experimental results on real-world datasets show that the proposed QREFINE method can generate refined questions with more readability but fewer mistakes than the original questions provided by users. Moreover, the refined questions also significantly improve the accuracy of answer retrieval.

arXiv commentCIKM 2019

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• Distributed Representations of Words and Phrases and their Compositionality
• Sequence to Sequence Learning with Neural Networks
• Playing Atari with Deep Reinforcement Learning
• Trust Region Policy Optimization
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
• An Actor-Critic Algorithm for Sequence Prediction
• Ask the Right Questions: Active Question Reformulation with Reinforcement Learning
• Neural Language Correction with Character-Based Attention
• Query-Reduction Networks for Question Answering
• Massive Exploration of Neural Machine Translation Architectures
• MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
• Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
• Learning to Paraphrase for Question Answering
• Paraphrase Generation with Deep Reinforcement Learning
• Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
• Proximal Policy Optimization and its Dynamic Version for Sequence Generation
• Identifying Well-formed Natural Language Questions