Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning

Abstract

Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In particular, the overhead of the state-of-the-art protocols for secure model aggregation grows quadratically with the number of users. In this paper, we propose the first secure aggregation framework, named Turbo-Aggregate, that in a network with $N$ users achieves a secure aggregation overhead of $O(N\log N)$, as opposed to $O(N^2)$, while tolerating up to a user dropout rate of $50\%$. Turbo-Aggregate employs a multi-group circular strategy for efficient model aggregation, and leverages additive secret sharing and novel coding techniques for injecting aggregation redundancy in order to handle user dropouts while guaranteeing user privacy. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to $40\times$ speedup over the state-of-the-art protocols with up to $N=200$ users. Our experiments also demonstrate the impact of model size and bandwidth on the performance of Turbo-Aggregate.

References in corpus (20)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• On the Convergence of FedAvg on Non-IID Data
• Learning Differentially Private Recurrent Language Models
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Differentially Private Federated Learning: A Client Level Perspective
• Applied Federated Learning: Improving Google Keyboard Query Suggestions
• Can You Really Backdoor Federated Learning?
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Fair Resource Allocation in Federated Learning
• Agnostic Federated Learning
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Practical Secure Aggregation for Federated Learning on User-Held Data
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Peer-to-peer Federated Learning on Graphs
• Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
• Central Server Free Federated Learning over Single-sided Trust Social Networks
• CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning
• COLA: Decentralized Linear Learning
• Client-Edge-Cloud Hierarchical Federated Learning

Cited by in corpus (8)

• FedML: A Research Library and Benchmark for Federated Machine Learning
• Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
• The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape
• Communication-Computation Efficient Secure Aggregation for Federated Learning
• Secure Shapley Value for Cross-Silo Federated Learning (Technical Report)
• A Scalable Approach for Privacy-Preserving Collaborative Machine Learning
• Byzantine-Resilient Secure Federated Learning
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX