Learning Modular Exponentiation with Transformers

David Demitri Africa, Sara M. Kapoor, Theo Simon Sorg +1

Modular exponentiation is crucial to number theory and cryptography, yet remains largely unexplored from a mechanistic interpretability standpoint. We train a 4-layer encoder-decod…