Interpreting and Steering State-Space Models via Activation Subspace Bottlenecks

Vamshi Sunku Mohan, Kaustubh Gupta, Aneesha Das +1

State-space models (SSMs) have emerged as an efficient strategy for building powerful language models, avoiding the quadratic complexity of computing attention in transformers. Des…