GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation
arXiv:2404.03746 · doi:10.1007/978-3-031-56063-7_24
Abstract
Query Reformulation(QR) is a set of techniques used to transform a user's original search query to a text that better aligns with the user's intent and improves their search experience. Recently, zero-shot QR has been shown to be a promising approach due to its ability to exploit knowledge inherent in large language models. By taking inspiration from the success of ensemble prompting strategies which have benefited many tasks, we investigate if they can help improve query reformulation. In this context, we propose an ensemble based prompting technique, GenQREnsemble which leverages paraphrases of a zero-shot instruction to generate multiple sets of keywords ultimately improving retrieval performance. We further introduce its post-retrieval variant, GenQREnsembleRF to incorporate pseudo relevant feedback. On evaluations over four IR benchmarks, we find that GenQREnsemble generates better reformulations with relative nDCG@10 improvements up to 18% and MAP improvements upto 24% over the previous zero-shot state-of-art. On the MSMarco Passage Ranking task, GenQREnsembleRF shows relative gains of 5% MRR using pseudo-relevance feedback, and 9% nDCG@10 using relevant feedback documents.
arXiv commentAccepted at ECIR 2024
References in corpus (6)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- Simplified Data Wrangling with ir_datasets
- Ask Me Anything: A simple strategy for prompting language models
- Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study