Federated Learning is Better with Non-Homomorphic Encryption

Abstract

Traditional AI methodologies necessitate centralized data collection, which becomes impractical when facing problems with network communication, data privacy, or storage capacity. Federated Learning (FL) offers a paradigm that empowers distributed AI model training without collecting raw data. There are different choices for providing privacy during FL training. One of the popular methodologies is employing Homomorphic Encryption (HE) - a breakthrough in privacy-preserving computation from Cryptography. However, these methods have a price in the form of extra computation and memory footprint. To resolve these issues, we propose an innovative framework that synergizes permutation-based compressors with Classical Cryptography, even though employing Classical Cryptography was assumed to be impossible in the past in the context of FL. Our framework offers a way to replace HE with cheaper Classical Cryptography primitives which provides security for the training process. It fosters asynchronous communication and provides flexible deployment options in various communication topologies.

arXiv comment56 pages, 10 figures, Accepted to presentation and proceedings to 4th ACM International Workshop on Distributed Machine Learning

References in corpus (17)

• TensorFlow: A system for large-scale machine learning
• One weird trick for parallelizing convolutional neural networks
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Gradient Sparsification for Communication-Efficient Distributed Optimization
• Beyond Data and Model Parallelism for Deep Neural Networks
• Natural Compression for Distributed Deep Learning
• Understanding Gradient Clipping in Private SGD: A Geometric Perspective
• FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning
• EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback
• Distributed learning with compressed gradients
• On Privacy and Personalization in Cross-Silo Federated Learning
• FL_PyTorch: optimization research simulator for federated learning
• Permutation Compressors for Provably Faster Distributed Nonconvex Optimization
• Revisiting Fully Homomorphic Encryption Schemes
• Faster Rates for Compressed Federated Learning with Client-Variance Reduction