End-to-End Retrieval in Continuous Space

Abstract

Most text-based information retrieval (IR) systems index objects by words or phrases. These discrete systems have been augmented by models that use embeddings to measure similarity in continuous space. But continuous-space models are typically used just to re-rank the top candidates. We consider the problem of end-to-end continuous retrieval, where standard approximate nearest neighbor (ANN) search replaces the usual discrete inverted index, and rely entirely on distances between learned embeddings. By training simple models specifically for retrieval, with an appropriate model architecture, we improve on a discrete baseline by 8% and 26% (MAP) on two similar-question retrieval tasks. We also discuss the problem of evaluation for retrieval systems, and show how to modify existing pairwise similarity datasets for this purpose.

References in corpus (6)

• Exploring the Limits of Language Modeling
• Natural Language Inference over Interaction Space
• Efficient Natural Language Response Suggestion for Smart Reply
• Towards Universal Paraphrastic Sentence Embeddings
• StarSpace: Embed All The Things!
• Quantization based Fast Inner Product Search

Cited by in corpus (20)

• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Distilling Knowledge from Reader to Retriever for Question Answering
• Self-supervised Learning for Large-scale Item Recommendations
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Neural Passage Retrieval with Improved Negative Contrast
• RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble
• MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
• Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO
• Learning Contextualized Document Representations for Healthcare Answer Retrieval
• LAReQA: Language-agnostic answer retrieval from a multilingual pool
• Talk, Don't Write: A Study of Direct Speech-Based Image Retrieval
• Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
• Transferable Representation Learning in Vision-and-Language Navigation
• Learning Federated Representations and Recommendations with Limited Negatives
• Text-to-Image Generation Grounded by Fine-Grained User Attention
• Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation
• Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization
• PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling