Why mask diffusion does not work

Haocheng Sun, Cynthia Xin Wen, Edward Hong Wang

The main advantages of diffusion language models over autoregressive (AR) models lie in their ability to support parallel generation and bidirectional attention, enabling a more co…