17 papers
Constrained Adaptive Rejection Sampling
PaweÅ Parys, Sairam Vaidya, Taylor Berg-Kirkpatrick +1
Language Models (LMs) are increasingly used in applications where generated outputs must satisfy strict semantic or syntactic constraints. Existing approaches to constrained genera…
Learning the Error Patterns of Language Models
Jinwoo Kim, Taylor Berg-KirkPatrick, Loris D'Antoni
When generating outputs for domains with specific validity constraints (e.g., a program should compile), LLMs often fail in a small number of focused ways: for example, by using Py…
Continuous Diffusion Models Can Obey Formal Syntax
Jinwoo Kim, Taylor Berg-Kirkpatrick, Loris D'Antoni
Diffusion language models offer a promising alternative to autoregressive models due to their global, non-causal generation process, but their continuous latent dynamics make discr…
Manifold-Guided Attention Steering
Ian Li, Kapilesh Guruprasad, Raunak Sengupta +3
Large language models frequently produce errors in reasoning tasks despite possessing the underlying knowledge required for correct reasoning. One possible approach to improve reas…
Language-Based Agent Control
Timothy Zhou, Loris D'Antoni, Nadia Polikarpova
This paper introduces language-based agent control (LBAC), a new programming model for agentic applications that brings techniques from programming languages and language-based sec…
The Format Tax
Ivan Yee Lee, Loris D'Antoni, Taylor Berg-Kirkpatrick
Asking a large language model to respond in JSON should be a formatting choice, not a capability tax. Yet we find that structured output requirements -- JSON, XML, LaTeX, Markdown…