Private Learning on Networks: Part II

Abstract

This paper considers a distributed multi-agent optimization problem, with the global objective consisting of the sum of local objective functions of the agents. The agents solve the optimization problem using local computation and communication between adjacent agents in the network. We present two randomized iterative algorithms for distributed optimization. To improve privacy, our algorithms add "structured" randomization to the information exchanged between the agents. We prove deterministic correctness (in every execution) of the proposed algorithms despite the information being perturbed by noise with non-zero mean. We prove that a special case of a proposed algorithm (called function sharing) preserves privacy of individual polynomial objective functions under a suitable connectivity condition on the network topology.

arXiv commentPrivacy-Convergence Trade-off added. New simulation results added (Current Version: 5 November 2017. First Version: 27 March 2017. )

References in corpus (11)

• Deep Learning with Differential Privacy
• On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Differentially Private Distributed Constrained Optimization
• Learning Privately from Multiparty Data
• Towards Geo-Distributed Machine Learning
• On the O(1/k) Convergence of Asynchronous Distributed Alternating Direction Method of Multipliers
• On the Linear Convergence of Distributed Optimization over Directed Graphs
• Private Learning on Networks
• Distributed Optimization of Convex Sum of Non-Convex Functions
• Deep Learning with Eigenvalue Decay Regularizer

Cited by in corpus (4)

• Minimizing Latency for Secure Coded Computing Using Secret Sharing via Staircase Codes
• Privacy-preserving Incremental ADMM for Decentralized Consensus Optimization
• ADMM Based Privacy-preserving Decentralized Optimization
• A Private and Finite-Time Algorithm for Solving a Distributed System of Linear Equations