4 papers
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies
Benjamin Clavié, Sean Lee, Aamir Shakir +1
We propose Latent Terms, a method revealing that models trained for dense retrieval, whether single- or multi-vector, learn representations that can trivially be decomposed into re…
IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval
Benjamin Clavié, Atoof Shakir, Jonah Turner +3
Multimodal Information Retrieval has made significant progress in recent years, leveraging the increasingly strong multimodal abilities of deep pre-trained models to represent info…
Fantastic (small) Retrievers and How to Train Them: mxbai-edge-colbert-v0 Tech Report
Rikiya Takehi, Benjamin Clavié, Sean Lee +1
In this work, we introduce mxbai-edge-colbert-v0 models, at two different parameter counts: 17M and 32M. As part of our research, we conduct numerous experiments to improve retriev…
Simple Projection Variants Improve ColBERT Performance
Benjamin Clavié, Sean Lee, Rikiya Takehi +2
Multi-vector dense retrieval methods like ColBERT systematically use a single-layer linear projection to reduce the dimensionality of individual vectors. In this study, we explore…