A Distributed One-Step Estimator

Abstract

Distributed statistical inference has recently attracted enormous attention. Many existing work focuses on the averaging estimator. We propose a one-step approach to enhance a simple-averaging based distributed estimator. We derive the corresponding asymptotic properties of the newly proposed estimator. We find that the proposed one-step estimator enjoys the same asymptotic properties as the centralized estimator. The proposed one-step approach merely requires one additional round of communication in relative to the averaging estimator; so the extra communication burden is insignificant. In finite sample cases, numerical examples show that the proposed estimator outperforms the simple averaging estimator with a large margin in terms of the mean squared errors. A potential application of the one-step approach is that one can use multiple machines to speed up large scale statistical inference with little compromise in the quality of estimators. The proposed method becomes more valuable when data can only be available at distributed machines with limited communication bandwidth.

31 pages

References in corpus (4)

• Communication-Efficient Distributed Dual Coordinate Ascent
• Improved Distributed Principal Component Analysis
• Distributed Estimation, Information Loss and Exponential Families
• Median Selection Subset Aggregation for Parallel Inference

Cited by in corpus (8)

• A review of distributed statistical inference
• Efficient Distributed Learning with Sparsity
• Distributed Inference for Linear Support Vector Machine
• Nonparametric Heterogeneity Testing For Massive Data
• Individual Data Protected Integrative Regression Analysis of High-dimensional Heterogeneous Data
• Simultaneous Inference for Massive Data: Distributed Bootstrap
• A Provably Communication-Efficient Asynchronous Distributed Inference Method for Convex and Nonconvex Problems
• Bootstrap Model Aggregation for Distributed Statistical Learning