Dishonesty in Helpful and Harmless Alignment

Youcheng Huang, Jingkun Tang, Duanyu Feng +4

People tell lies when seeking rewards. Large language models (LLMs) are aligned to human values with reinforcement learning where they get rewards if they satisfy human preference.…