Federated learning with differential privacy and an untrusted aggregator

Abstract

Federated learning for training models over mobile devices is gaining popularity. Current systems for this task exhibit significant trade-offs between model accuracy, privacy guarantee, and device efficiency. For instance, Oort (OSDI 2021) provides excellent accuracy and efficiency but requires a trusted central server. On the other hand, Orchard (OSDI 2020) provides good accuracy and the rigorous guarantee of differential privacy over an untrusted server, but creates huge overhead for the devices. This paper describes Aero, a new federated learning system that significantly improves this trade-off. Aero guarantees good accuracy, differential privacy over an untrusted server, and keeps the device overhead low. The key idea of Aero is to tune system architecture and design to a specific set of popular, federated learning algorithms. This tuning requires novel optimizations and techniques, e.g., a new protocol to securely aggregate updates from devices. An evaluation of Aero demonstrates that it provides comparable accuracy to plain federated learning (without differential privacy), and it improves efficiency (CPU and network) over Orchard by up to $10^5\times$.

22 pages, 10 figures, published in ICISSP 2024

References in corpus (16)

• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• Prio: Private, Robust, and Scalable Computation of Aggregate Statistics
• Efficient Dropout-resilient Aggregation for Privacy-preserving Machine Learning
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning
• FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning
• Federated Learning for Ranking Browser History Suggestions
• Secure Federated Submodel Learning
• LDP-Fed: Federated Learning with Local Differential Privacy
• Federated Dynamic GNN with Secure Aggregation
• Sparsified Secure Aggregation for Privacy-Preserving Federated Learning
• Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors
• EIFFeL: Ensuring Integrity for Federated Learning
• Protecting Data from all Parties: Combining FHE and DP in Federated Learning
• On the Information Theoretic Secure Aggregation with Uncoded Groupwise Keys
• SecFL: Confidential Federated Learning using TEEs
• Encoded Gradients Aggregation against Gradient Leakage in Federated Learning