Intermediate Distillation: Data-Efficient Distillation from Black-Box LLMs for Information Retrieval

Zizhong Li, Haopeng Zhang, Jiawei Zhang

Recent research has explored distilling knowledge from large language models (LLMs) to optimize retriever models, especially within the retrieval-augmented generation (RAG) framewo…