From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling

Mohsinul Kabir, Tasfia Tahsin, Sophia Ananiadou

Current research on bias in language models (LMs) predominantly focuses on data quality, with significantly less attention paid to model architecture and temporal influences of dat…