The Power of Noise: Redefining Retrieval for RAG Systems

Abstract

Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of Large Language Models by augmenting the original prompt with relevant passages or documents retrieved by an Information Retrieval (IR) system. RAG has become increasingly important for Generative AI solutions, especially in enterprise settings or in any domain in which knowledge is constantly refreshed and cannot be memorized in the LLM. We argue here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, we conduct the first comprehensive and systematic examination of the retrieval strategy of RAG systems. We focus, in particular, on the type of passages IR systems within a RAG solution should retrieve. Our analysis considers multiple factors, such as the relevance of the passages included in the prompt context, their position, and their number. One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM. Even more surprising, we discovered that adding random documents in the prompt improves the LLM accuracy by up to 35%. These results highlight the need to investigate the appropriate strategies when integrating retrieval with LLMs, thereby laying the groundwork for future research in this area.

References in corpus (1)

• Multimodal Neural Databases

Cited by in corpus (14)

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge
• Automated Review Generation Method Based on Large Language Models
• An overview of domain-specific foundation model: key technologies, applications and challenges
• "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval"
• Securing RAG: A Risk Assessment and Mitigation Framework
• A New HOPE: Domain-agnostic Automatic Evaluation of Text Chunking
• Adaptive Orchestration of Modular Generative Information Access Systems
• Advanced System Integration: Analyzing OpenAPI Chunking for Retrieval-Augmented Generation
• Constructing and Evaluating Declarative RAG Pipelines in PyTerrier
• RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation
• CoMaPOI: A Collaborative Multi-Agent Framework for Next POI Prediction Bridging the Gap Between Trajectory and Language
• Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
• The Hitchhikers Guide to Production-ready Trustworthy Foundation Model powered Software (FMware)