Agent Step Value: Auditing Evaluator-Channel Reversals in Black-Box Agent Traces
arXiv:2607.04419
The paper introduces Agent Step Value (ASV), a metric for auditing how step‑wise rewards assigned by an evaluator can change sign when the scoring channel changes, and demonstrates its use on a large set of PubMed question‑answering transitions.
Abstract
Pooling, substituting, or reusing evaluator-derived step rewards assumes that their direction survives a change of evaluation channel. The same frozen transition can violate that assumption. Process rewards vary agent states, while evaluator audits vary scoring configurations; neither first difference isolates their interaction. We define Agent Step Value (ASV) as a channel-indexed target-margin gain and identify the state-by-channel interaction on complete matched faces. Across frozen PubMed question-answering transitions, direct scoring yields a positive mean ASV, while the generated-view channel yields a negative mean. Two matched replay waves reproduce this reversal, and cross-channel sign disagreement exceeds same-channel retry disagreement by 48.0 percentage points. Matched retrieval faces localize the reversal to the generated-view coordinate and trace its direction across a readout-and-stack bridge. A source-only generation contract restores the positive mean direction on artifact-bearing retrievals and removes parser-detected substantive support claims from artifact-free before-state views. ASV turns channel sensitivity into an identified measurement problem that can be localized and tested by intervention before step rewards are reused.
adds source-contract intervention and two-wave retry study; re-acquires cube and extends bridge cohort to n=98 to 100