Recontextualization Mitigates Specification Gaming without Modifying the Specification

Ariana Azarbal, Victor Gillioz, Vladimir Ivanov +6

Developers often struggle to specify correct training labels and rewards. Perhaps they don't need to. We propose recontextualization, which reduces how often language models "game"…