Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
arXiv:2103.09593
Abstract
Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix when conversing with each other. Inspired by this phenomenon, we present two strong black-box adversarial attacks (one word-level, one phrase-level) for multilingual models that push their ability to handle code-mixed sentences to the limit. The former uses bilingual dictionaries to propose perturbations and translations of the clean example for sense disambiguation. The latter directly aligns the clean example with its translations before extracting phrases as perturbations. Our phrase-level attack has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. Finally, we propose an efficient adversarial training scheme that trains in the same number of steps as the original model and show that it improves model accuracy.
arXiv commentTo be presented at NAACL-HLT 2021. Abstract also published in the Rising Stars Track of the Workshop on Computational Approaches to Linguistic Code-Switching (CALCS 2021)
References in corpus (11)
- On the Cross-lingual Transferability of Monolingual Representations
- Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding
- MASS: Masked Sequence to Sequence Pre-training for Language Generation
- BAE: BERT-based Adversarial Examples for Text Classification
- Cross-Lingual Ability of Multilingual BERT: An Empirical Study
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
- It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
- A Survey of Code-switched Speech and Language Processing
- Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset
- Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data
- Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi