SANE: State Anomaly Neutralization for Stable Extreme-Context Delta-Rule Models

Qingwen Lin, Boyan Xu, Xiao Liu +2

Delta-Rule recurrent models maintain a fixed-size state, enabling $O(1)$ inference memory but potentially becoming unstable under extreme-context extrapolation. By tracking RWKV-7…