Reward Weighted Classifier-Free Guidance as Policy Improvement in Autoregressive Models

Alexander Peysakhovich, William Berman

Consider an auto-regressive model that produces outputs x (e.g., answers to questions, molecules) each of which can be summarized by an attribute vector y (e.g., helpfulness vs. ha…