Contrastive Retrieval Heads Improve Attention-Based Re-Ranking

Linh Tran, Yulong Li, Radu Florian +1

The strong zero-shot and long-context capabilities of recent Large Language Models (LLMs) have paved the way for highly effective re-ranking systems. Attention-based re-rankers lev…