Next-token Preference: Scaling Unsupervised Reward Modeling

Jingxuan Fan, Yueying Li, Zhenting Qi +4

Learning from feedback advances the capabilities and safety of frontier models, but preference supervision remains costly and difficult to scale. We introduce next-token preference…