Optimal rates and adaptation in the single-index model using aggregation

Abstract

We want to recover the regression function in the single-index model. Using an aggregation algorithm with local polynomial estimators, we answer in particular to the second part of Question~2 from Stone (1982) on the optimal convergence rate. The procedure constructed here has strong adaptation properties: it adapts both to the smoothness of the link function and to the unknown index. Moreover, the procedure locally adapts to the distribution of the design. We propose new upper bounds for the local polynomial estimator (which are results of independent interest) that allows a fairly general design. The behavior of this algorithm is studied through numerical simulations. In particular, we show empirically that it improves strongly over empirical risk minimization.

36 pages

Cited by in corpus (11)

• Sparse Regression Learning by Aggregation and Langevin Monte-Carlo
• User-friendly introduction to PAC-Bayes bounds
• Sparse single-index model
• Nonparametric estimation of composite functions
• Adaptive estimation under single-index constraint in a regression model
• On the optimality of the aggregate with exponential weights for low temperatures
• Semiparametric Efficiency in Convexity Constrained Single Index Model
• Nonconvex sparse regularization for deep neural networks and its optimality
• Aggregation of penalized empirical risk minimizers in regression
• Recovery of regular ridge functions on the ball
• Learning and adaptive estimation for marker-dependent counting processes