GRADE: Replacing Policy Gradients with Backpropagation for LLM Alignment

Lukas Abrie Nel

Reinforcement learning from human feedback (RLHF) has become the dominant paradigm for aligning large language models with human preferences. However, policy gradient methods such…