Federated learning with differential privacy and an untrusted aggregator

Abstract

Federated learning for training models over mobile devices is gaining popularity. Current systems for this task exhibit significant trade-offs between model accuracy, privacy guarantee, and device efficiency. For instance, Oort (OSDI 2021) provides excellent accuracy and efficiency but requires a trusted central server. On the other hand, Orchard (OSDI 2020) provides good accuracy and the rigorous guarantee of differential privacy over an untrusted server, but creates huge overhead for the devices. This paper describes Aero, a new federated learning system that significantly improves this trade-off. Aero guarantees good accuracy, differential privacy over an untrusted server, and keeps the device overhead low. The key idea of Aero is to tune system architecture and design to a specific set of popular, federated learning algorithms. This tuning requires novel optimizations and techniques, e.g., a new protocol to securely aggregate updates from devices. An evaluation of Aero demonstrates that it provides comparable accuracy to plain federated learning (without differential privacy), and it improves efficiency (CPU and network) over Orchard by up to $10^5\times$.

arXiv comment22 pages, 10 figures, published in ICISSP 2024

References in corpus (31)

• Deep Learning with Differential Privacy
• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• Federated Learning for Mobile Keyboard Prediction
• Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
• Learning Differentially Private Recurrent Language Models
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Collecting and Analyzing Data from Smart Device Users with Local Differential Privacy
• Prio: Private, Robust, and Scalable Computation of Aggregate Statistics
• Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning
• Efficient Deep Learning on Multi-Source Private Data
• FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning
• PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments
• Secure Byzantine-Robust Machine Learning
• FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
• Federated Learning for Ranking Browser History Suggestions
• Secure Federated Submodel Learning
• LDP-Fed: Federated Learning with Local Differential Privacy
• Federated Dynamic GNN with Secure Aggregation
• Sparsified Secure Aggregation for Privacy-Preserving Federated Learning
• Secure Weighted Aggregation for Federated Learning
• Byzantine-Robust and Privacy-Preserving Framework for FedML
• Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors
• EIFFeL: Ensuring Integrity for Federated Learning
• Efficient Sparse Secure Aggregation for Federated Learning
• Protecting Data from all Parties: Combining FHE and DP in Federated Learning
• On the Information Theoretic Secure Aggregation with Uncoded Groupwise Keys
• SecFL: Confidential Federated Learning using TEEs
• Encoded Gradients Aggregation against Gradient Leakage in Federated Learning