Let your LLM generate a few tokens and you will reduce the need for retrieval

Hervé Déjean

In this paper, we investigate how efficiently large language models (LLM) can be trained to check whether an answer is already stored in their parametric memory. We distill an LLM-…