The Pre-Training Study of Expanded-SPLADE Models on Web Document Titles

Hiun Kim, Tae Kwan Lee, Taeryun Won

Masked Language Modeling (MLM) pre-training is one of the primary ways to initialize Neural Information Retrieval (IR) models prior to retrieval fine-tuning. However, studies show…