Generalization Limits of Reinforcement Learning Alignment

Haruhi Shida, Koo Imai, Keigo Kansa

The safety of large language models (LLMs) relies on alignment techniques such as reinforcement learning from human feedback (RLHF). However, recent theoretical analyses suggest th…