Characterizing Mamba's Selective Memory using Auto-Encoders

Tamanna Hossain, Robert L. Logan, Ganesh Jagadeesan +3

State space models (SSMs) are a promising alternative to transformers for language modeling because they use fixed memory during inference. However, this fixed memory usage require…