Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting

Zahra Fatemi, Chen Xing, Wenhao Liu +1

Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such…