When Disagreement Labels Do Not Identify F1 Rankings

Roberto Fernández-Barrios, Iker Pastor-López, Asier González-Santocildes +1

Agreement labels cancel from an accuracy comparison but can change an F1 ranking. We specialize partial identification to an explicit audit after labeling disagreements: two agreem…