Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
arXiv:2012.14862
Abstract
The effectiveness of Neural Information Retrieval (Neu-IR) often depends on a large scale of in-domain relevance training signals, which are not always available in real-world ranking scenarios. To democratize the benefits of Neu-IR, this paper presents MetaAdaptRank, a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains. Drawing on source-domain massive relevance supervision, MetaAdaptRank contrastively synthesizes a large number of weak supervision signals for target domains and meta-learns to reweight these synthetic "weak" data based on their benefits to the target-domain ranking accuracy of Neu-IR models. Experiments on three TREC benchmarks in the web, news, and biomedical domains show that MetaAdaptRank significantly improves the few-shot ranking accuracy of Neu-IR models. Further analyses indicate that MetaAdaptRank thrives from both its contrastive weak data synthesis and meta-reweighted data selection. The code and data of this paper can be obtained from https://github.com/thunlp/MetaAdaptRank.
arXiv comment14 pages, accepted by ACL-IJCNLP 2021 (long paper)
References in corpus (12)
- A Deep Relevance Matching Model for Ad-hoc Retrieval
- Learning to Reweight Examples for Robust Deep Learning
- End-to-End Neural Ad-hoc Ranking with Kernel Pooling
- Deeper Text Understanding for IR with Contextual Neural Language Modeling
- Passage Re-ranking with BERT
- Evaluation of Text Generation: A Survey
- Understanding the Behaviors of BERT in Ranking
- Word-Entity Duet Representations for Document Ranking
- Learning Data Manipulation for Augmentation and Weighting
- Overview of the TREC 2021 deep learning track
- Content-Based Weak Supervision for Ad-Hoc Re-Ranking
- OpenMatch: An Open Source Library for Neu-IR Research