First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers

Arjun Pillai, Christian Hoang, Anjelo Jann Laroza

Why do multilingual language models sometimes generate in the wrong language, and why is this so hard to fix? We introduce Language Identity Head Ablation (LIHA), a causal interven…