Communication-efficient sparse regression: a one-shot approach

Abstract

We devise a one-shot approach to distributed sparse regression in the high-dimensional setting. The key idea is to average "debiased" or "desparsified" lasso estimators. We show the approach converges at the same rate as the lasso as long as the dataset is not split across too many machines. We also extend the approach to generalized linear models.

29 pages, 3 figures

References in corpus (4)

• On Communication Cost of Distributed Statistical Estimation and Dimensionality
• Optimality guarantees for distributed statistical estimation
• A General Framework for Robust Testing and Confidence Regions in High-Dimensional Quantile Regression
• Median Selection Subset Aggregation for Parallel Inference

Cited by in corpus (17)

• Distributed Estimation and Inference with Statistical Guarantees
• Least Squares Approximation for a Distributed System
• Efficient Distributed Learning with Sparsity
• Communication-Efficient Distributed Statistical Inference
• Distributed Multi-Task Learning with Shared Representation
• Communication-efficient Distributed Sparse Linear Discriminant Analysis
• Debiased distributed learning for sparse partial linear models in high dimensions
• Nonparametric Heterogeneity Testing For Massive Data
• A Distributed One-Step Estimator
• Efficient Distributed Estimation of Inverse Covariance Matrices
• MSplit LBI: Realizing Feature Selection and Dense Estimation Simultaneously in Few-shot and Zero-shot Learning
• Distributed Multitask Learning
• Communication-efficient Distributed Estimation and Inference for Transelliptical Graphical Models
• Distributed Community Detection for Large Scale Networks Using Stochastic Block Model
• A Sequential Addressing Subsampling Method for Massive Data Analysis under Memory Constraint
• Distributed Pseudo-Likelihood Method for Community Detection in Large-Scale Networks
• Communication-efficient Algorithm for Distributed Sparse Learning via Two-way Truncation