Distributed Learning for Cooperative Inference

Abstract

We study the problem of cooperative inference where a group of agents interact over a network and seek to estimate a joint parameter that best explains a set of observations. Agents do not know the network topology or the observations of other agents. We explore a variational interpretation of the Bayesian posterior density, and its relation to the stochastic mirror descent algorithm, to propose a new distributed learning algorithm. We show that, under appropriate assumptions, the beliefs generated by the proposed algorithm concentrate around the true parameter exponentially fast. We provide explicit non-asymptotic bounds for the convergence rate. Moreover, we develop explicit and computationally efficient algorithms for observation models belonging to exponential families.

References in corpus (4)

• Linear Time Average Consensus on Fixed Graphs and Implications for Decentralized Optimization and Multi-Agent Control
• Provable Bayesian Inference via Particle Mirror Descent
• Bayesian Posteriors Without Bayes' Theorem
• Distributed Gaussian Learning over Time-varying Directed Graphs

Cited by in corpus (6)

• A Field Guide to Federated Optimization
• Optimal Algorithms for Distributed Optimization
• Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenters
• On Primal-Dual Approach for Distributed Stochastic Convex Optimization over Networks
• First-order Methods with Convergence Rates for Multi-agent Systems on Semidefinite Matrix Spaces
• Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors