Characterizing the Behavior of Training Mamba-based State Space Models on GPUs

Trinayan Baruah, Kaustubh Shivdikar, Sara Prescott +1

Mamba-based State Space Models (SSM) have emerged as a promising alternative to the ubiquitous transformers. Despite the expressive power of transformers, the quadratic complexity…