6 papers · 1 filter
Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
Pascal Jutras-Dubé, Patrick Pynadath, Jeremy Lu +2
We introduce Sticky Jump Diffusions (SJDs), continuous-time Markov processes on whose discrete anchors are token embeddings. In forward time, anchors release their ma…
CANDI: Hybrid Discrete-Continuous Diffusion Models
Patrick Pynadath, Jiaxin Shi, Ruqi Zhang
While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete for…
Generative Frontiers: Why Evaluation Matters for Diffusion Language Models
Patrick Pynadath, Jiaxin Shi, Ruqi Zhang
Diffusion language models have seen exciting recent progress, offering far more flexibility in generative trajectories than autoregressive models. This flexibility has motivated a…
Why Any-Order Autoregressive Models Need Two-Stream Attention: A Structural-Semantic Tradeoff
Patrick Pynadath, Ruqi Zhang
Any-order autoregressive models (AO-ARMs) offer a promising path toward efficient masked diffusion by enabling native key-value caching, but competitive performance has so far requ…
Single-Step Consistent Diffusion Samplers
Pascal Jutras-Dubé, Patrick Pynadath, Ruqi Zhang
Sampling from unnormalized target distributions is a fundamental yet challenging task in machine learning and statistics. Existing sampling algorithms typically require many iterat…
Gradient-based Discrete Sampling with Automatic Cyclical Scheduling
Patrick Pynadath, Riddhiman Bhattacharya, Arun Hariharan +1
Discrete distributions, particularly in high-dimensional deep models, are often highly multimodal due to inherent discontinuities. While gradient-based discrete sampling has proven…