Heterogeneous Adaptive Policy Optimization: Tailoring Optimization to Every Token's Nature

Zheng Liu, Mengjie Liu, Siwei Wen +4

Using entropy as a measure of heterogeneity to guide optimization has emerged as a crucial research direction in Reinforcement Learning for LLMs. However, existing methods typicall…