Neural Vector Spaces for Unsupervised Information Retrieval

Abstract

We propose the Neural Vector Space Model (NVSM), a method that learns representations of documents in an unsupervised manner for news article retrieval. In the NVSM paradigm, we learn low-dimensional representations of words and documents from scratch using gradient descent and rank documents according to their similarity with query representations that are composed from word representations. We show that NVSM performs better at document ranking than existing latent semantic vector space methods. The addition of NVSM to a mixture of lexical language models and a state-of-the-art baseline vector space model yields a statistically significant increase in retrieval effectiveness. Consequently, NVSM adds a complementary relevance signal. Next to semantic matching, we find that NVSM performs well in cases where lexical matching is needed. NVSM learns a notion of term specificity directly from the document collection without feature engineering. We also show that NVSM learns regularities related to Luhn significance. Finally, we give advice on how to deploy NVSM in situations where model selection (e.g., cross-validation) is infeasible. We find that an unsupervised ensemble of multiple models trained with different hyperparameter values performs better than a single cross-validated model. Therefore, NVSM can safely be used for ranking documents without supervised relevance judgments.

arXiv commentTOIS 2018

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Efficient Estimation of Word Representations in Vector Space
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Natural Language Processing (almost) from Scratch
• Distributed Representations of Sentences and Documents
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• Learning Latent Vector Spaces for Product Search
• Noisy Activation Functions
• Efficient Vector Representation for Documents through Corruption
• Unsupervised, Efficient and Semantic Expertise Retrieval
• Fast k Nearest Neighbor Search using GPU
• Off the Beaten Path: Let's Replace Term-Based Retrieval with k-NN Search
• Reply With: Proactive Recommendation of Email Attachments
• Semantic Entity Retrieval Toolkit

Cited by in corpus (14)

• Information Retrieval: Recent Advances and Beyond
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Critically Examining the "Neural Hype": Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models
• Zero-Shot Heterogeneous Transfer Learning from Recommender Systems to Cold-Start Search Retrieval
• Leveraging Semantic and Lexical Matching to Improve the Recall of Document Retrieval Systems: A Hybrid Approach
• A Deep Look into Neural Ranking Models for Information Retrieval
• Dataset search: a survey
• Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
• Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
• Remedies against the Vocabulary Gap in Information Retrieval
• Unsupervised Low-Dimensional Vector Representations for Words, Phrases and Text that are Transparent, Scalable, and produce Similarity Metrics that are Complementary to Neural Embeddings
• ILPS at TREC 2017 Common Core Track
• Multilingual Music Genre Embeddings for Effective Cross-Lingual Music Item Annotation
• SAUCE: Truncated Sparse Document Signature Bit-Vectors for Fast Web-Scale Corpus Expansion