A Communication-efficient Local Differentially Private Algorithm in Federated Optimization

Abstract

Federated optimization, wherein several agents in a network collaborate with a central server to achieve optimal social cost over the network with no requirement for exchanging information among agents, has attracted significant interest from the research community. In this context, agents demand resources based on their local computation. Due to the exchange of optimization parameters such as states, constraints, or objective functions with a central server, an adversary may infer sensitive information of agents. We develop a differentially-private additive-increase and multiplicative-decrease algorithm to allocate multiple divisible shared heterogeneous resources to agents in a network. The developed algorithm provides a differential privacy guarantee to each agent in the network. The algorithm does not require inter-agent communication, and the agents do not need to share their cost function or their derivatives with other agents or a central server; however, they share their allocation states with a central server that keeps track of the aggregate consumption of resources. The algorithm incurs very little communication overhead; for m heterogeneous resources in the system, the asymptotic upper bound on the communication complexity is O(m) bits at a time step. Furthermore, if the algorithm converges in K time steps, then the upper bound communication complexity will be O(mK) bits. The algorithm can find applications in several areas, including smart cities, smart energy systems, resource management in the sixth generation (6G) wireless networks with privacy guarantees, etc. We present experimental results to check the efficacy of the algorithm. Furthermore, we present empirical analyses for the trade-off between privacy and algorithm efficiency.

References in corpus (18)

• Deep Learning with Differential Privacy
• The Future of Digital Health with Federated Learning
• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• Federated Learning for Mobile Keyboard Prediction
• Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Differentially Private Distributed Constrained Optimization
• Collecting Telemetry Data Privately
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• A Field Guide to Federated Optimization
• Federated Learning for Emoji Prediction in a Mobile Keyboard
• Comprehensive Introduction to Fully Homomorphic Encryption for Dynamic Feedback Controller via LWE-based Cryptosystem
• Federated Composite Optimization
• Federated $f$-Differential Privacy
• Distributed Algorithms for Internet-of-Things-enabled Prosumer Markets: A Control Theoretic Perspective
• Derandomized Distributed Multi-resource Allocation with Little Communication Overhead