Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHF

Simeng Sun, Dhawal Gupta, Mohit Iyyer

During the last stage of RLHF, a large language model is aligned to human intents via PPO training, a process that generally requires large-scale computational resources. In this t…