Reward Generalization in RLHF: A Topological Perspective

Tianyi Qiu, Fanzhi Zeng, Jiaming Ji +7

Existing alignment methods share a common topology of information flow, where reward information is collected from humans, modeled with preference learning, and used to tune langua…