Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval

Abstract

Learned sparse document representations using a transformer-based neural model has been found to be attractive in both relevance effectiveness and time efficiency. This paper describes a representation sparsification scheme based on hard and soft thresholding with an inverted index approximation for faster SPLADE-based document retrieval. It provides analytical and experimental results on the impact of this learnable hybrid thresholding scheme.

This paper is published in SIGIR'23

References in corpus (9)

• Overview of the TREC 2020 deep learning track
• An Efficiency Study for SPLADE Models
• A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques
• Optimizing Guided Traversal for Fast Learned Sparse Retrieval
• From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective
• Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrieval
• Sparsifying Sparse Representations for Passage Retrieval by Top-$k$ Masking
• Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
• Dual Skipping Guidance for Document Retrieval with Learned Sparse Representations