Metropolis-Hastings Dominates Importance Resampling for Policy Composition

Alexey Kurennoy, Ramil Yarullin, Fergal Reid

Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive. Decoding-time policy compos…