P${}^3$LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training

Junwei Bao, Yifan Wang, Jiangyong Ying +4

Conventional autoregressive left-to-right (L2R) sequence generation faces two issues during decoding: limited to unidirectional target sequence modeling, and constrained on strong…