Training-Free Exponential Context Extension via Cascading KV Cache

Jeffrey Willette, Heejun Lee, Youngwan Lee +2

The transformer's context window is vital for tasks such as few-shot learning and conditional generation as it preserves previous tokens for active memory. However, as the context…