Diffusion On Syntax Trees For Program Synthesis

Shreyas Kapur, Erik Jenner, Stuart Russell

Large language models generate code one token at a time. Their autoregressive generation process lacks the feedback of observing the program's output. Training LLMs to suggest edit…