Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters
arXiv:2012.13176
Abstract
Multilingual Neural Machine Translation architectures mainly differ in the amount of sharing modules and parameters among languages. In this paper, and from an algorithmic perspective, we explore if the chosen architecture, when trained with the same data, influences the gender bias accuracy. Experiments in four language pairs show that Language-Specific encoders-decoders exhibit less bias than the Shared encoder-decoder architecture. Further interpretability analysis of source embeddings and the attention shows that, in the Language-Specific case, the embeddings encode more gender information, and its attention is more diverted. Both behaviors help in mitigating gender bias.
arXiv comment12 pages, 5 figures, 3 tables
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Sequence to Sequence Learning with Neural Networks
- Measuring and Reducing Gendered Correlations in Pre-trained Models
- Evaluating the Underlying Gender Bias in Contextualized Word Embeddings
- Analyzing Gender Stereotyping in Bollywood Movies
- Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
- Generating Clues for Gender based Occupation De-biasing in Text