Neural Estimation of Pairwise Mutual Information in Masked Discrete Sequence Models

Jai Sharma, Yifan Wang, Bryan Li

Understanding dependencies between variables is critical for interpretability and efficient generation in masked diffusion models (MDMs), yet these models primarily expose marginal…