Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments

Abstract

Interactive decision support systems often require repeated preference judgments, although identifying the best alternative may not justify eliciting all \(n(n-1)/2\) pairwise comparisons. We propose ALEC (Adaptive Lexicographic Elicitation of Comparisons), which re-ranks the alternatives after each judgment and asks the first unobserved pair in the lexicographic order of the current ranking, using the Analytic Hierarchy Process as the concrete setting. Under a log-normal error model, directly comparing a contrast maximises the one-step reduction in its effective resistance. Posterior bounds on the probability of a wrong choice or of exceeding a relative-loss tolerance provide stopping rules valid under adaptive questioning, while a consistency-and-margin condition gives a deterministic certificate of agreement with the full-matrix winner. Simulations show that re-ranking is most useful in larger problems with limited budgets; ALEC outperforms Harker's original rule in most scenarios with \(n\ge11\), while under a common stopping rule it achieves higher hit-rate point estimates than the stronger Harker-log variant at a modest additional questioning cost. With a 5\% loss tolerance and a cap of \(2n\) additional comparisons, the stopping rule saves \res{prot-s06-saveadd-min}--\res{prot-s06-saveadd-max}\% of them, while among runs stopped by the bound the observed rate of losses above tolerance remains at most \res{s06-errearly-max}\%, against a nominal 10\%. ALEC therefore provides an adaptive elicitation module for decision support systems that balances decision precision against user effort.