Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere

Mikaela Martins, Jonathan Dunn

Previous work has argued that multilingual LLMs encode language-specific features most strongly in their initial and final layers, implying that intermediate layers support languag…