Candidate-Label Saliency for Efficient Conformal Pruning of Text Classifiers

Abstract

Recalibration can preserve conformal coverage after pruning while enlarging prediction sets and leaving more alternatives for downstream decisions. Directly searching for deletions that keep sets small is costly. We study whether a small candidate-gradient budget can approximate this search at lower cost. Candidate-Saliency Pruning (CSP) estimates the squared first-order change in two plausible labels' scores when each parameter is deleted. CSP averages these changes, combines the resulting importance with a base pruner, and selects the mixing weight under validation accuracy and coverage requirements. Independent split-conformal calibration supplies marginal coverage under exchangeability. We evaluate CSP on three classification tasks, two base pruners, and three sparsity levels against a squared true-label control that separates candidate choice from gradient squaring, under fixed recovery updates, equal total time, and restricted access to the training corpus. Before recovery, CSP reduces mean set size by 13.2\% to 19.2\% against the reference pruner and by 7.7\% to 13.2\% against the control in all six encoder cells at 50\% sparsity. The advantage over the control shrinks with recovery to practical equivalence in all six cells at 1,000 updates, and at equal total time the cheaper reference pruner is practically equivalent in 18 of 18 settings. When recovery is limited to the retained pruning pool, CSP keeps a meaningful advantage in all six cells. Its value therefore depends on the time and data available for recovery.

arXiv comment32 pages, 4 figures