Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models

Xinyu Liu, Runsong Zhao, Pengcheng Huang +5

Numerous recent works target to extend effective context length for language models and various methods, tasks and benchmarks exist to measure model's effective memorization length…