6 papers
A Continuous-Time Markov Chain Framework for Insertion Language Models
Dhruvesh Patel, Benjamin Rozonoyer, Soumitra Das +3
Insertion Language Models (ILMs) offer several advantages over left-to-right generation and mask-based generation. However, existing formulations of insertion-based generation have…
Insertion Based Sequence Generation with Learnable Order Dynamics
Dhruvesh Patel, Benjamin Rozonoyer, Gaurav Pandey +3
Existing insertion-based masked diffusion models that generate sequences by interleaving token insertion with unmasking use fixed schedules that are not dependent on the data. For…
Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
Keshav Ramji, Tahira Naseem, Ramón Fernandez Astudillo
While long, explicit chains-of-thought (CoT) have proven effective on complex reasoning tasks, they are costly to generate during inference. Non-verbal reasoning methods have emerg…
Latent Principle Discovery for Language Model Self-Improvement
Keshav Ramji, Tahira Naseem, Ramón Fernandez Astudillo
When language model (LM) users aim to improve the quality of its generations, it is crucial to specify concrete behavioral attributes that the model should strive to reflect. Howev…
Optimal Policy Minimum Bayesian Risk
Ramón Fernandez Astudillo, Md Arafat Sultan, Aashka Trivedi +4
Inference scaling helps LLMs solve complex reasoning problems through extended runtime computation. On top of long chain-of-thought (long-CoT) models, purely inference-time techniq…
Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions
Dhruvesh Patel, Aishwarya Sahoo, Avinash Amballa +3
Autoregressive models (ARMs), which predict subsequent tokens one-by-one ``from left to right,'' have achieved significant success across a wide range of sequence generation tasks.…