Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift

Zitong Huang, Gustavo Lucas Carvalho, Deqing Fu +1

We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task. Our key intuition is th…