COBRA: A Combined Regression Strategy

Abstract

A new method for combining several initial estimators of the regression function is introduced. Instead of building a linear or convex optimized combination over a collection of basic estimators $r_1,\dots ,r_M$, we use them as a collective indicator of the proximity between the training data and a test observation. This local distance approach is model-free and very fast. More specifically, the resulting nonparametric/nonlinear combined estimator is shown to perform asymptotically at least as well in the $L^2$ sense as the best combination of the basic estimators in the collective. A companion R package called \cobra (standing for COmBined Regression Alternative) is presented (downloadable on \url{http://cran.r-project.org/web/packages/COBRA/index.html}). Substantial numerical evidence is provided on both synthetic and real data sets to assess the excellent performance and velocity of our method in a large variety of prediction problems.

42 pages

References in corpus (3)

• Analysis of a Random Forests Model
• High-dimensional additive modeling
• Sparse single-index model

Cited by in corpus (8)

• Pycobra: A Python Toolbox for Ensemble Learning and Visualisation
• Non-linear aggregation of filters to improve image denoising
• SPOCC: Scalable POssibilistic Classifier Combination -- toward robust aggregation of classifiers
• Kernel-Based Ensemble Learning in Python
• Accelerated Gradient Boosting
• A clusterwise supervised learning procedure based on aggregation of distances
• An optimal aggregation type classifier
• A combined strategy for multivariate density estimation