Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau +1

Muon-trained modular-arithmetic transformers can lose accuracy while retaining linearly decodable task information. Adjacent swaps localize five captured unnormalized failures to A…