Sharp Oracle Inequalities for Low-complexity Priors
arXiv:1702.03166
Abstract
In this paper,we consider a high-dimensional statistical estimation problem in which the the number of parameters is comparable or larger than the sample size. We present a unified analysis of the performance guarantees of exponential weighted aggregation and penalized estimators with a general class of data losses and priors which encourage objects which conform to some notion of simplicity/complexity. More precisely, we show that these two estimators satisfy sharp oracle inequalities for prediction ensuring their good theoretical performances. We also highlight the differences between them. When the noise is random, we provide oracle inequalities in probability using concentration inequalities. These results are then applied to several instances including the Lasso, the group Lasso, their analysis-type counterparts, the and the nuclear norm penalties. All our estimators can be efficiently implemented using proximal splitting algorithms.
arXiv commenthal-01422476 version available 26 Dec. 2016
References in corpus (8)
- Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
- Simultaneous analysis of Lasso and Dantzig selector
- Consistency of the group Lasso and multiple kernel learning
- SLOPE - Adaptive variable selection via convex optimization
- Smoothing proximal gradient method for general structured sparse regression
- On the Prediction Performance of the Lasso
- Simultaneous adaptation to the margin and to complexity in classification
- On the Exponentially Weighted Aggregate with the Laplace Prior