Memory Determines Learning Direction: A Theory of Gradient-Based Optimization in State Space Models

JingChuan Guan, Tomoyuki Kubota, Yasuo Kuniyoshi +1

State space models (SSMs) have gained attention by showing potential to outperform Transformers. However, previous studies have not sufficiently addressed the mechanisms underlying…