Efficient Perplexity Bound and Ratio Matching in Discrete Diffusion Language Models

Etrit Haxholli, Yeti Z. Gurbuz, Ogul Can +1

While continuous diffusion models excel in modeling continuous distributions, their application to categorical data has been less effective. Recent work has shown that ratio-matchi…