Anatomy of an Idiom: Tracing Non-Compositionality in Language Models

Andrew Gomes

We investigate the processing of idiomatic expressions in transformer-based language models using a novel set of techniques for circuit discovery and analysis. First discovering ci…