TADA: Task-Agnostic Dialect Adapters for English

Abstract

Large Language Models, the dominant starting point for Natural Language Processing (NLP) applications, fail at a higher rate for speakers of English dialects other than Standard American English (SAE). Prior work addresses this using task-specific data or synthetic data augmentation, both of which require intervention for each dialect and task pair. This poses a scalability issue that prevents the broad adoption of robust dialectal English NLP. We introduce a simple yet effective method for task-agnostic dialect adaptation by aligning non-SAE dialects using adapters and composing them with task-specific adapters from SAE. Task-Agnostic Dialect Adapters (TADA) improve dialectal robustness on 4 dialectal variants of the GLUE benchmark without task-specific supervision.

arXiv comment5 Pages; ACL Findings Paper 2023

References in corpus (7)

• On the Opportunities and Risks of Foundation Models
• Cross-lingual Language Model Pretraining
• mT5: A massively multilingual pre-trained text-to-text transformer
• What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
• CALCS 2021 Shared Task: Machine Translation for Code-Switched Data
• Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
• CST5: Data Augmentation for Code-Switched Semantic Parsing