10 papers · 1 filter
The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding
Işıl Özgü, Yaoxuan Wu, Guy Van den Broeck +1
Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking dis…
Mitigating Bias in Locally Constrained Decoding via Tractable Proposals
Meihua Dang, Linxin Song, Honghua Zhang +3
Generations from large language models often fail to conform to desired constraints such as JSON schema. Existing locally constrained decoding (LCD) approaches enforce constraints…
Probabilistic Programs of Thought
Poorva Garg, Renato Lui Geh, Daniel Israel +3
LLMs are widely used for code generation and mathematical reasoning tasks where they are required to generate structured output. They either need to reason about code, generate cod…
Learning Tractable Distributions Of Language Model Continuations
Gwen Yidou-Weng, Ian Li, Anji Liu +4
Controlled generation imposes sequence-level constraints (syntax, style, safety) that depend on future tokens, making exact conditioning of an autoregressive LM intractable. Tracta…
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Daniel Israel, Guy Van den Broeck, Aditya Grover
The generation speed of LLMs are bottlenecked by autoregressive decoding, where tokens are predicted sequentially one by one. Alternatively, diffusion large language models (dLLMs)…
TRACE Back from the Future: A Probabilistic Reasoning Approach to Controllable Language Generation
Gwen Yidou Weng, Benjie Wang, Guy Van den Broeck
As large language models (LMs) advance, there is an increasing need to control their outputs to align with human values (e.g., detoxification) or desired attributes (e.g., personal…