Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

Abstract

While question answering (QA) with neural network, i.e. neural QA, has achieved promising results in recent years, lacking of large scale real-word QA dataset is still a challenge for developing and evaluating neural QA system. To alleviate this problem, we propose a large scale human annotated real-world QA dataset WebQA with more than 42k questions and 556k evidences. As existing neural QA methods resolve QA either as sequence generation or classification/ranking problem, they face challenges of expensive softmax computation, unseen answers handling or separate candidate answer generation component. In this work, we cast neural QA as a sequence labeling problem and propose an end-to-end sequence labeling model, which overcomes all the above challenges. Experimental results on WebQA show that our model outperforms the baselines significantly with an F1 score of 74.69% with word-based input, and the performance drops only 3.72 F1 points with more challenging character-based input.

10 pages, 3 figures, withdraw experimental results on CNN/Daily Mail datasets

References in corpus (6)

• Improving neural networks by preventing co-adaptation of feature detectors
• Large-scale Simple Question Answering with Memory Networks
• Attentive Pooling Networks
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• Separating Answers from Queries for Neural Reading Comprehension
• Machine Comprehension Based on Learning to Rank

Cited by in corpus (25)

• ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
• QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
• Learning Recurrent Span Representations for Extractive Question Answering
• PanGu-$\alpha$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
• Automatic Rule Extraction from Long Short Term Memory Networks
• Embracing data abundance: BookTest Dataset for Reading Comprehension
• DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
• Making Neural QA as Simple as Possible but not Simpler
• Towards a Better Understanding Human Reading Comprehension with Brain Signals
• A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
• Cross-Lingual Natural Language Generation via Pre-Training
• s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning
• Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension
• Multimodal deep learning for short-term stock volatility prediction
• SRQA: Synthetic Reader for Factoid Question Answering
• Knowledge Graph Question Answering via SPARQL Silhouette Generation
• Contextual Recurrent Units for Cloze-style Reading Comprehension
• Dependent Gated Reading for Cloze-Style Question Answering
• CalibreNet: Calibration Networks for Multilingual Sequence Labeling
• Wizard of Search Engine: Access to Information Through Conversations with Search Engines
• Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning
• A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension
• No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
• Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension
• Dual Attention Network for Product Compatibility and Function Satisfiability Analysis