Coverage Improvement and Fast Convergence of On-policy Preference Learning

Juno Kim, Jihun Yun, Jason D. Lee +1

Online on-policy preference learning algorithms for language model alignment such as online direct policy optimization (DPO) can significantly outperform their offline counterparts…