Dense Hierarchical Retrieval for Open-Domain Question Answering
arXiv:2110.15439
Abstract
Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA), where latent representations of questions and passages are exploited for maximum inner product search in the retrieval process. However, current dense retrievers require splitting documents into short passages that usually contain local, partial, and sometimes biased context, and highly depend on the splitting process. As a consequence, it may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework that can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage. Specifically, a document-level retriever first identifies relevant documents, among which relevant passages are then retrieved by a passage-level retriever. The ranking of the retrieved passages will be further calibrated by examining the document-level relevance. In addition, hierarchical title structure and two negative sampling strategies (i.e., In-Doc and In-Sec negatives) are investigated. We apply DHR to large-scale open-domain QA datasets. DHR significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperform the strong baselines on multiple open-domain QA benchmarks.
arXiv commentEMNLP 2021 Findings
References in corpus (7)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
- Pre-training Tasks for Embedding-based Large-scale Retrieval
- Generation-Augmented Retrieval for Open-domain Question Answering
- Neural Passage Retrieval with Improved Negative Contrast
- Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering