Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need

Zhengyi Ma, Zhicheng Dou, Wei Xu +4

Designing pre-training objectives that more closely resemble the downstream tasks for pre-trained language models can lead to better performance at the fine-tuning stage, especiall…