Investigating the Indirect Object Identification circuit in Mamba

Danielle Ensign, Adrià Garriga-Alonso

How well will current interpretability techniques generalize to future models? A relevant case study is Mamba, a recent recurrent architecture with scaling comparable to Transforme…