Entropy-Guided Greedy Heuristic for Minimal Differential Set Construction

Abstract

This paper addresses the challenge of identifying a minimal subset of discrete predictor variables that best explains a binary class. We propose a greedy heuristic for feature selection that sequentially constructs a differential set by iteratively adding the variable that yields the most significant reduction in conditional entropy. The method incorporates a confidence-guided acceptance criterion that accounts for finite-sample uncertainty, allowing the search to adaptively balance the risk of including spurious variables against the opportunity to capture meaningful predictors. Because the procedure yields a compact set of statistically validated predictors together with an explicit confidence level for each one, it is well suited to healthcare innovation and management applications such as biomarker and risk-factor identification, diagnostic panel design, and clinical decision support, where minimizing the number of tests or indicators collected is critical for cost, feasibility, and patient burden.

arXiv comment20 pages, 6 figures