Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance Margins

Lukas Gienapp, Niklas Deckers, Martin Potthast +1

Representation-based retrieval models, so-called bi-encoders, estimate the relevance of a document to a query by calculating the similarity of their respective embeddings. Current…