6 papers
A Continuous-Time Markov Chain Framework for Insertion Language Models
Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3
Insertion Language Models (ILMs) offer several advantages over left-to-right generation and mask-based generation. However, existing formulations of insertion-based generation have…
Learned Relay Representations for Forward-Thinking Discrete Diffusion Models
Benjamin Rozonoyer, Jacopo Minniti, Dhruvesh Patel +4
When Masked Diffusion Models (MDMs) generate sequences through iterative refinement, the rich internal computation over masked positions is discarded, forcing every subsequent refi…
Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders
Benjamin Rozonoyer, Chong You, Michael Boratko +5
The success of Large Language Models (LLMs) has motivated a shift toward generative approaches to retrieval and ranking, aiming to supersede classical Dual Encoders (DEs) and Cross…
Insertion Based Sequence Generation with Learnable Order Dynamics
Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey +3
Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data. For…
XLM: A Python package for non-autoregressive language models
Dhruvesh Patel, Durga Prasad Maram, Sai Sreenivas Chintha +2
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressi…
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation
Jiachen Zhao, Wenlong Zhao, Andrew Drozdov +5
We study semi-supervised sequence generation tasks, where the few labeled examples are too scarce to finetune a model, and meanwhile, few-shot prompted large language models (LLMs)…