Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models

Philipp Mondorf, Sondre Wold, Barbara Plank

A fundamental question in interpretability research is to what extent neural networks, particularly language models, implement reusable functions through subnetworks that can be co…