HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs

Darsh Kachroo, Arjun Prasaath Anbazhagan, Adriana Caraeni +2

Direct Preference Optimization (DPO) is an effective framework for aligning large language models with human preferences, but it struggles with complex reasoning tasks. DPO optimiz…