Generative Relevance Feedback with Large Language Models
arXiv:2304.13157 · doi:10.1145/3539618.3591992
Abstract
Current query expansion models use pseudo-relevance feedback to improve first-pass retrieval effectiveness; however, this fails when the initial results are not relevant. Instead of building a language model from retrieved results, we propose Generative Relevance Feedback (GRF) that builds probabilistic feedback models from long-form text generated from Large Language Models. We study the effective methods for generating text by varying the zero-shot generation subtasks: queries, entities, facts, news articles, documents, and essays. We evaluate GRF on document retrieval benchmarks covering a diverse set of queries and document collections, and the results show that GRF methods significantly outperform previous PRF methods. Specifically, we improve MAP between 5-19% and NDCG@10 17-24% compared to RM3 expansion, and achieve the best R@1k effectiveness on all datasets compared to state-of-the-art sparse, dense, and expansion models.
arXiv commentSIGIR 2023 Preprint, 6 pages
References in corpus (12)
- Training language models to follow instructions with human feedback
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- PaLM: Scaling Language Modeling with Pathways
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- Unsupervised Dense Information Retrieval with Contrastive Learning
- Overview of the TREC 2020 deep learning track
- Distilling Dense Representations for Ranking using Tightly-Coupled Teachers
- InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrieval
- Naver Labs Europe (SPLADE) @ TREC Deep Learning 2022
- Query Expansion Using Contextual Clue Sampling with Language Models
- ExaRanker: Explanation-Augmented Neural Ranker
- IntenT5: Search Result Diversification using Causal Language Models