2D Stability Selection: Design Jittering to Exclude Correlated Irrelevant Features

Abstract

Stability Selection resamples the observations but leaves the design untouched, so it cannot reject an irrelevant feature that is correlated with the relevant ones in a way that violates the irrepresentable condition. Such a feature (a ``confounder'') is selected in a substantial fraction of resamples, and the exchangeability behind the false-positive bound of Stability Selection fails. We propose 2D Stability Selection (2D-SS), which aggregates selection frequencies over half-samples and over a grid of injected design noise, and reduces to Stability Selection at zero noise. Because input noise acts, in expectation, as Tikhonov regularisation, a jittered Lasso fit is a noisy elastic net whose support recovery is governed by a noise-indexed irrepresentable condition. Jittering can therefore repair designs in which confounders violate the Lasso condition, although it can also hurt. We give the noise level at which a violated feature's statistic vanishes (for uncorrelated relevant features), exact-recovery guarantees for a single fit, conditions under which aggregation over noise levels recovers the support, and a complementary-pairs bound on the expected number of false positives, carried over from Stability Selection to the noise-aggregated procedure, for a fixed threshold or a size-matched selector. The procedure takes no measurement-error covariance as input, although the guarantees assume i.i.d.\ Gaussian measurement error. In simulations 2D-SS excludes confounders that Stability Selection includes, and its size-matched version stays below the nominal closed-form false-positive bound in all 240 simulated cells, whereas Stability Selection exceeds it in 21; it does not generally improve on Stability Selection in $F_1$ and is lower at low noise. It is useful when stable but wrong selections are the concern.