Trapped by simplicity: When Transformers fail to learn from noisy features

Evan Peters, Ando Deng, Matheus H. Zambianco +2

Noise is ubiquitous in data used to train large language models, but it is not well understood whether these models are able to correctly generalize to inputs generated without noi…