Mixture-of-Memories: Low-Overhead Language Modeling with Modular Knowledge Injection

Yifan Lu, Rubin Wei, Ziyun Zou +8

Large language models store substantial factual knowledge in their parameters, yet this knowledge is difficult to control, update, and reliably elicit at inference time. Existing m…