4 papers
Insertion Based Sequence Generation with Learnable Order Dynamics
Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey +3
Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data. For…
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4
When Masked Diffusion Models (MDMs) generate sequences through iterative refinement, the rich internal computation over masked positions is discarded, forcing every subsequent refi…
XLM: A Python package for non-autoregressive language models
Dhruvesh Patel, Durga Prasad Maram, Sai Sreenivas Chintha +2
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressi…
Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions
Dhruvesh Patel, Aishwarya Sahoo, Avinash Amballa +3
Autoregressive models (ARMs), which predict subsequent tokens one-by-one ``from left to right,'' have achieved significant success across a wide range of sequence generation tasks.…