False Sense of Safety in Selective Signal Classification: Auditing Bound Tightness and Exchangeability for Risk Control

Jingwen Zhou, Mingzhe Wang

Selective prediction with distribution-free risk control promises that, with confidence 1-delta over the calibration draw, the error rate of accepted inputs stays below a user budg…