Investigating the Effects of Sparse Attention on Cross-Encoders

Ferdinand Schlatt, Maik Fröbe, Matthias Hagen

Cross-encoders are effective passage and document re-rankers but less efficient than other neural or classic retrieval models. A few previous studies have applied windowed self-att…