SocraticPO: Policy Optimization via Interactive Guidance

Zirui Liu, Jie Ouyang, Qi Liu +8

Reinforcement learning (RL) for large language models usually supervises reasoning with scalar outcome rewards, such as binary correctness. Such rewards provide an optimization dir…