Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs New Metrics

Antonio Franca, Alexander Tong

Diffusion language models (DLMs) have emerged as the leading non-autoregressive alternative to language modeling. Because their likelihoods are intractable, generative perplexity (…