Optimal Algorithms for Ridge and Lasso Regression with Partially Observed Attributes

Abstract

We consider the most common variants of linear regression, including Ridge, Lasso and Support-vector regression, in a setting where the learner is allowed to observe only a fixed number of attributes of each example at training time. We present simple and efficient algorithms for these problems: for Lasso and Ridge regression they need the same total number of attributes (up to constants) as do full-information algorithms, for reaching a certain accuracy. For Support-vector regression, we require exponentially less attributes compared to the state of the art. By that, we resolve an open problem recently posed by Cesa-Bianchi et al. (2010). Experiments show the theoretical bounds to be justified by superior performance compared to the state of the art.

This is a full version of the paper arXiv:1206.4678 appearing in ICML 2012

References in corpus (2)

• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• Efficient Learning with Partially Observed Attributes

Cited by in corpus (4)

• Efficient Learning with Partially Observed Attributes
• Linear Regression with Limited Observation
• Passive Learning with Target Risk
• Exploiting Smoothness in Statistical Learning, Sequential Prediction, and Stochastic Optimization