RAGDoll: Efficient Offloading-based Online RAG System on a Single GPU

Weiping Yu, Ningyi Liao, Siqiang Luo +1

Retrieval-Augmented Generation (RAG) enhances large language model (LLM) generation quality by incorporating relevant external knowledge. However, deploying RAG on consumer-grade p…