Mamba Modulation: On the Length Generalization of Mamba

Peng Lu, Jerry Huang, Qiuhao Zeng +4

The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space…