Complementary Roles of Activation and Parametric Memory in Few-Shot Learning

Miaohe Niu, Runsong Zhao, Xinyu Liu +5

At test time, large language models (LLMs) can encode historical information in activation memory (i.e., KV caches) and parametric memory (i.e., updated parameters). While activati…