SocraticPO: Policy Optimization via Interactive Guidance

Zirui Liu, Tingyue Pan, Jie Ouyang +8

Reinforcement learning (RL) for large language models usually supervises reasoning with scalar outcome rewards, such as binary correctness. Such rewards provide an optimization dir…