Where Should a Document Live: Context, Representations, or Parameters?

Nathanaël Carraz Rakotonirina, Momchil Hardalov, Gonzalo Iglesias +1

To answer questions outside of their pre-training data, large language models (LLMs) need access to new information, which can be presented in the context window as documents, enco…