Learning Tractable Distributions Of Language Model Continuations

Gwen Yidou-Weng, Ian Li, Anji Liu +4

Controlled generation imposes sequence-level constraints (syntax, style, safety) that depend on future tokens, making exact conditioning of an autoregressive LM intractable. Tracta…