Communication-Efficient Distributed Statistical Inference
arXiv:1605.07689
Abstract
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation.
References in corpus (8)
- An MCMC Approach to Classical Estimation
- Communication Efficient Distributed Optimization using an Approximate Newton-type Method
- Parallelizing MCMC via Weierstrass Sampler
- Optimality guarantees for distributed statistical estimation
- Communication-efficient sparse regression: a one-shot approach
- Efficient Distributed Learning with Sparsity
- Asynchronous Gibbs Sampling
- Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares