Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning

Ciarán M. Gilligan-Lee, Joseph Egan, Yuchen Zhu +1

Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors -- such as misaligned pers…