Composite Re-Ranking for Efficient Document Search with BERT
arXiv:2103.06499 · doi:10.1145/3488560.3498495
Abstract
Although considerable efforts have been devoted to transformer-based ranking models for document search, the relevance-efficiency tradeoff remains a critical problem for ad-hoc ranking. To overcome this challenge, this paper presents BECR (BERT-based Composite Re-Ranking), a composite re-ranking scheme that combines deep contextual token interactions and traditional lexical term-matching features. In particular, BECR exploits a token encoding mechanism to decompose the query representations into pre-computable uni-grams and skip-n-grams. By applying token encoding on top of a dual-encoder architecture, BECR separates the attentions between a query and a document while capturing the contextual semantics of a query. In contrast to previous approaches, this framework does not perform expensive BERT computations during online inference. Thus, it is significantly faster, yet still able to achieve high competitiveness in ad-hoc ranking relevance. Finally, an in-depth comparison between BECR and other start-of-the-art neural ranking baselines is described using the TREC datasets, thereby further demonstrating the enhanced relevance and efficiency of BECR.
arXiv commentto be published in WSDM'22
References in corpus (6)
- A Deep Relevance Matching Model for Ad-hoc Retrieval
- End-to-End Neural Ad-hoc Ranking with Kernel Pooling
- Q8BERT: Quantized 8Bit BERT
- Deeper Text Understanding for IR with Contextual Neural Language Modeling
- CEDR: Contextualized Embeddings for Document Ranking
- DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling