A necessary and sufficient stability notion for adaptive generalization
arXiv:1906.00930
Abstract
We introduce a new notion of the stability of computations, which holds under post-processing and adaptive composition. We show that the notion is both necessary and sufficient to ensure generalization in the face of adaptivity, for any computations that respond to bounded-sensitivity linear queries while providing accuracy with respect to the data sample set. The stability notion is based on quantifying the effect of observing a computation's outputs on the posterior over the data sample elements. We show a separation between this stability notion and previously studied notion and observe that all differentially private algorithms also satisfy this notion.
References in corpus (6)
- Information-theoretic analysis of generalization capability of learning algorithms
- On the `Semantics' of Differential Privacy: A Bayesian Formulation
- Generalization in Adaptive Data Analysis and Holdout Reuse
- Learners that Use Little Information
- A New Approach to Adaptive Data Analysis and Learning via Maximal Leakage
- Natural Analysts in Adaptive Data Analysis