Language Model Activations Inhabit Privileged Error-Correcting Basins

Matthew Finlayson, Francisco Pernice, Eric Todd +16

Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize t…