How Does Diffusion Influence Pretrained Language Models on Out-of-Distribution Data?

Huazheng Wang, Daixuan Cheng, Haifeng Sun +5

Transformer-based pretrained language models (PLMs) have achieved great success in modern NLP. An important advantage of PLMs is good out-of-distribution (OOD) robustness. Recently…