Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules

Karim Galliamov, Rochelle Choenni, Ivan Titov

LLMs encode both general capabilities and domain-specific knowledge in a single set of parameters. We ask whether this capacity can be reorganized: keeping broadly useful computati…