On the Convergence of Self-Improving Online LLM Alignment

Xudong Wu, Pangpang Liu, Vaneet Aggarwal +1

The Self-Improving Alignment (SAIL) algorithm addresses distribution shift by reducing a bilevel formulation of the problem to an efficient, single-level method. Empirically, SAIL…