5 papers
A Continuous-Time Markov Chain Framework for Insertion Language Models
Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3
Insertion Language Models (ILMs) offer several advantages over left-to-right generation and mask-based generation. However, existing formulations of insertion-based generation have…
Insertion Based Sequence Generation with Learnable Order Dynamics
Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey +3
Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data. For…
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4
When Masked Diffusion Models (MDMs) generate sequences through iterative refinement, the rich internal computation over masked positions is discarded, forcing every subsequent refi…
Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders
Benjamin Rozonoyer, Chong You, Michael Boratko +5
The success of Large Language Models (LLMs) has motivated a shift toward generative approaches to retrieval and ranking, aiming to supersede classical Dual Encoders (DEs) and Cross…
XLM: A Python package for non-autoregressive language models
Dhruvesh Patel, Durga Prasad Maram, Sai Sreenivas Chintha +2
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressi…