An Efficiency Study for SPLADE Models

Abstract

Latency and efficiency issues are often overlooked when evaluating IR models based on Pretrained Language Models (PLMs) in reason of multiple hardware and software testing scenarios. Nevertheless, efficiency is an important part of such systems and should not be overlooked. In this paper, we focus on improving the efficiency of the SPLADE model since it has achieved state-of-the-art zero-shot performance and competitive results on TREC collections. SPLADE efficiency can be controlled via a regularization factor, but solely controlling this regularization has been shown to not be efficient enough. In order to reduce the latency gap between SPLADE and traditional retrieval systems, we propose several techniques including L1 regularization for queries, a separation of document/query encoders, a FLOPS-regularized middle-training, and the use of faster query encoders. Our benchmark demonstrates that we can drastically improve the efficiency of these models while increasing the performance metrics on in-domain data. To our knowledge, {we propose the first neural models that, under the same computing constraints, \textit{achieve similar latency (less than 4ms difference) as traditional BM25}, while having \textit{similar performance (less than 10\% MRR@10 reduction)} as the state-of-the-art single-stage neural rankers on in-domain data}.

arXiv commentAccepted at SIGIR 2022 and ReNeuIR. Huggingface models available at https://huggingface.co/naver and code at https://github.com/naver/splade

Cited by in corpus (13)

• Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations
• Foundations of Vector Retrieval
• An Approximate Algorithm for Maximum Inner Product Search over Streaming Sparse Vectors
• Optimizing Guided Traversal for Fast Learned Sparse Retrieval
• Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval
• Pairing Clustered Inverted Indexes with kNN Graphs for Fast Approximate Retrieval over Learned Sparse Representations
• Systematic Evaluation of Neural Retrieval Models on the Touché 2020 Argument Retrieval Subset of BEIR
• Bridging Dense and Sparse Maximum Inner Product Search
• Neural Passage Quality Estimation for Static Pruning
• Effective Inference-Free Retrieval for Learned Sparse Representations
• Reproducing NevIR: Negation in Neural Information Retrieval
• Exploring ${\ell }_0$ Sparsification for Inference-free Sparse Retrievers
• Adaptive Sparsity Optimization with Learnable Soft Top-K and Per-Term Thresholding for Efficient Retrieval