Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generation

Zizhong Li, Haopeng Zhang, Jiawei Zhang

Retrieval-augmented generation framework can address the limitations of large language models by enabling real-time knowledge updates for more accurate answers. An efficient way in…