Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
arXiv:2405.07920 · doi:10.1007/978-3-031-88714-7_31
Abstract
Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do not match the effectiveness of their teacher LLMs. We hypothesize that this effectiveness gap is due to the fact that previous work has not applied the best-suited methods for fine-tuning cross-encoders on manually labeled data (e.g., hard-negative sampling, deep sampling, and listwise loss functions). To close this gap, we create a new dataset, Rank-DistiLLM. Cross-encoders trained on Rank-DistiLLM achieve the effectiveness of LLMs while being up to 173 times faster and 24 times more memory efficient. Our code and data is available at https://github.com/webis-de/ECIR-25.
Accepted at ECIR'25
References in corpus (9)
- Array Programming with NumPy
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- The Information Retrieval Experiment Platform
- RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!
- RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models
- Lightning IR: Straightforward Fine-tuning and Inference of Transformer-based Language Models for Information Retrieval
- RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses
- Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Models