Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventions

Beiming Liu, Minjie Chen

Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural networks. Recent work showed th…