Discovering Interpretable Algorithms by Decompiling Transformers to RASP

Xinting Huang, Aleksandra Bakalova, Satwik Bhattamishra +2

Recent work has shown that the computations of Transformers can be simulated in the RASP family of programming languages. These findings have enabled improved understanding of the…