Rethinking Exposure Bias In Language Modeling

Yifan Xu, Kening Zhang, Haoyu Dong +3

Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned…