Linear Regression with Limited Observation

Abstract

We consider the most common variants of linear regression, including Ridge, Lasso and Support-vector regression, in a setting where the learner is allowed to observe only a fixed number of attributes of each example at training time. We present simple and efficient algorithms for these problems: for Lasso and Ridge regression they need the same total number of attributes (up to constants) as do full-information algorithms, for reaching a certain accuracy. For Support-vector regression, we require exponentially less attributes compared to the state of the art. By that, we resolve an open problem recently posed by Cesa-Bianchi et al. (2010). Experiments show the theoretical bounds to be justified by superior performance compared to the state of the art.

ICML2012

References in corpus (2)

• High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity
• Optimal Algorithms for Ridge and Lasso Regression with Partially Observed Attributes

Cited by in corpus (9)

• An Equivalence between the Lasso and Support Vector Machines
• Classification with Low Rank and Missing Data
• Randomized Block Subgradient Methods for Convex Nonsmooth and Stochastic Optimization
• Autoregressive-Model-Based Methods for Online Time Series Prediction with Missing Values: an Experimental Evaluation
• Active Structure Learning of Bayesian Networks in an Observational Setting
• Coupled Compound Poisson Factorization
• Adaptive Feature Selection: Computationally Efficient Online Sparse Linear Regression under RIP
• Data-Driven Online Decision Making with Costly Information Acquisition
• Online Sparse Linear Regression