Xetrieval: Explaining Dense Retrieval through Feature Elicitation

Abstract

Dense retrieval enables effective semantic matching, yet the semantic factors underlying its relevance scores remain opaque. Sparse autoencoders (SAEs) support feature-level explanations by decomposing dense embeddings into sparse semantic features. However, the document semantics underlying a retriever's relevance scores may not be sufficiently explicit in the embeddings to be recovered as interpretable SAE features. We introduce Xetrieval, a framework for explaining dense retrieval through feature elicitation. Feature Elicitors make retrieval-relevant aspects of document meaning more explicit through auxiliary views, facilitating their identification as interpretable SAE features. A shared sparse dictionary learns from these varied representations to express query--document relations through describable features. We assess their influence on relevance scores through interventions on the original document embeddings. Experiments show that Xetrieval yields more interpretable features than an SAE baseline and identifies feature sets with a stronger influence on the retriever's scores. Task-level feature modulation further demonstrates more effective retrieval steering. These findings highlight the value of feature elicitation for relevance attribution and retrieval steering.

arXiv commentCode: https://github.com/Hihiczx/Xetrieval ; Project page: https://hihiczx.github.io/Xetrieval