Mixture-of-Experts Language Models Can Be Strong and Efficient Retrievers

Anubhav Shrestha, Safal Shrestha, Minwu Kim +2

Recent work has shown that fine-tuning decoder-only large language models (LLMs) for retrieval yields strong first-stage retrievers, with effectiveness improving as backbones grow…