Training on the Test Model: Contamination in Ranking Distillation

Vishakha Suresh Kalal, Andrew Parry, Sean MacAvaney

Neural approaches to ranking based on pre-trained language models are highly effective in ad-hoc search. However, the computational expense of these models can limit their applicat…