Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training?

Tejas Dahiya, Cole Blondin

Mechanistic interpretability usually studies fully trained models, yet the computations that drive a behaviour can change while the model is still learning the task. On the Indirec…