Distributed High-dimensional Regression Under a Quantile Loss Function

Abstract

This paper studies distributed estimation and support recovery for high-dimensional linear regression model with heavy-tailed noise. To deal with heavy-tailed noise whose variance can be infinite, we adopt the quantile regression loss function instead of the commonly used squared loss. However, the non-smooth quantile loss poses new challenges to high-dimensional distributed estimation in both computation and theoretical development. To address the challenge, we transform the response variable and establish a new connection between quantile regression and ordinary linear regression. Then, we provide a distributed estimator that is both computationally and communicationally efficient, where only the gradient information is communicated at each iteration. Theoretically, we show that, after a constant number of iterations, the proposed estimator achieves a near-oracle convergence rate without any restriction on the number of machines. Moreover, we establish the theoretical guarantee for the support recovery. The simulation analysis is provided to demonstrate the effectiveness of our method.

arXiv comment42 pages, 5 figures

References in corpus (8)

• Nearly unbiased variable selection under minimax concave penalty
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Least squares after model selection in high-dimensional sparse models
• Composite quantile regression and the oracle Model Selection Theory
• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• Quantile Regression Under Memory Constraint
• Loss minimization and parameter estimation with heavy tails
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression

Cited by in corpus (5)

• First-order Newton-type Estimator for Distributed Estimation and Inference
• A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques
• An Asymptotic Analysis of Minibatch-Based Momentum Methods for Linear Regression Models
• On the asymptotic properties of a bagging estimator with a massive dataset
• Distributed Pseudo-Likelihood Method for Community Detection in Large-Scale Networks