TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents

Debajoy Mukherjee, Jeremy Carleton, Sushil Vemuri +2

Outcome-supervised group-relative policy optimization (GRPO) assigns the same advantage to every token in an LLM agent trajectory, leaving useful and unhelpful decisions undifferen…