Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards

Alexander G. Padula, Dennis J. N. J. Soemers

Proximal Policy Optimization (PPO) is commonly used in Reinforcement Learning from Human Feedback to align large language models (LLMs) with downstream tasks. This paper investigat…