MoNe: Modular Neural Memory for Efficient Long Context Inference

Wonguk Cho, Kyubyung Chae, Tribhuvanesh Orekondy +6

We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retraining. MoNe reads context in f…