Target-Token Asymmetry Can Masquerade As Depth Specialization

Danlin Li, Miao Li, Xu Han +1

Layer-ablation studies often perturb early or late layers of a language model, compare the damage to prediction on two corpora, and read a larger early-layer effect on one corpus a…