Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling

Benjamin Lipkin, Benjamin LeBrun, Jacob Hoover Vigly +9

The dominant approach to generating from language models subject to some constraint is locally constrained decoding (LCD), incrementally sampling tokens at each time step such that…