Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
arXiv:2008.03917
Abstract
Search engine has become a fundamental component in various web and mobile applications. Retrieving relevant documents from the massive datasets is challenging for a search engine system, especially when faced with verbose or tail queries. In this paper, we explore a vector space search framework for document retrieval. Specifically, we trained a deep semantic matching model so that each query and document can be encoded as a low dimensional embedding. Our model was trained based on BERT architecture. We deployed a fast k-nearest-neighbor index service for online serving. Both offline and online metrics demonstrate that our method improved retrieval performance and search quality considerably, particularly for tail
9 pages
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Distributed Representations of Words and Phrases and their Compositionality
- End-to-End Neural Ad-hoc Ranking with Kernel Pooling
- Query expansion techniques for information retrieval: A survey
- Understanding the Behaviors of BERT in Ranking
- Applying Deep Learning To Airbnb Search
- Neural Models for Information Retrieval
- A Study of MatchPyramid Models on Ad-hoc Retrieval
- Techniques for Deep Query Understanding