Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias

Yarden Tal, Inbal Magar, Roy Schwartz

The size of pretrained models is increasing, and so is their performance on a variety of NLP tasks. However, as their memorization capacity grows, they might pick up more social bi…