Neurocache: Efficient Vector Retrieval for Long-range Language Modeling

Ali Safaya, Deniz Yuret

This paper introduces Neurocache, an approach to extend the effective context size of large language models (LLMs) using an external vector cache to store its past states. Like rec…