Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations

Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos

A concept can carry different associations across languages, while modern language models learn English alongside many other languages during pretraining. Yet comparisons among exi…