Federated Learning: A Signal Processing Perspective

Abstract

The dramatic success of deep learning is largely due to the availability of data. Data samples are often acquired on edge devices, such as smart phones, vehicles and sensors, and in some cases cannot be shared due to privacy considerations. Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. Learning in a federated manner differs from conventional centralized machine learning, and poses several core unique challenges and requirements, which are closely related to classical problems studied in the areas of signal processing and communications. Consequently, dedicated schemes derived from these areas are expected to play an important role in the success of federated learning and the transition of deep learning from the domain of centralized servers to mobile edge devices. In this article, we provide a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools. We present a formulation for the federated learning paradigm from a signal processing perspective, and survey a set of candidate approaches for tackling its unique challenges. We further provide guidelines for the design and adaptation of signal processing and communication methods to facilitate federated learning at large scale.

arXiv comment24 pages, 15 figures

References in corpus (28)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Challenges, Methods, and Future Directions
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Learning for Mobile Keyboard Prediction
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• On the Convergence of FedAvg on Non-IID Data
• Towards Federated Learning at Scale: System Design
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
• Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation
• Three Approaches for Personalization with Applications to Federated Learning
• Multi-Armed Bandit Based Client Scheduling for Federated Learning
• Energy-Efficient Resource Allocation in Wireless Networks: An Overview of Game-Theoretic Approaches
• UVeQFed: Universal Vector Quantization for Federated Learning
• Agnostic Federated Learning
• Sparsified SGD with Memory
• Over-the-Air Federated Learning from Heterogeneous Data
• On Analog Gradient Descent Learning over Multiple Access Fading Channels
• Generalized Byzantine-tolerant SGD
• Robust Federated Learning in a Heterogeneous Environment
• Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
• Nomographic Functions: Efficient Computation in Clustered Gaussian Sensor Networks
• Multiple Source Adaptation and the Renyi Divergence
• Model-Based Deep Learning
• Bayesian Federated Learning over Wireless Networks
• Federated Dynamic Spectrum Access

Cited by in corpus (21)

• Over-the-Air Federated Learning from Heterogeneous Data
• Scheduling and Aggregation Design for Asynchronous Federated Learning over Wireless Networks
• STAR-RIS Integrated Non-Orthogonal Multiple Access and Over-the-Air Federated Learning: Framework, Analysis, and Optimization
• Integrating Over-the-Air Federated Learning and Non-Orthogonal Multiple Access: What Role can RIS Play?
• Joint Privacy Enhancement and Quantization in Federated Learning
• Graph Federated Learning for CIoT Devices in Smart Home Applications
• Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference
• Device Heterogeneity in Federated Learning: A Superquantile Approach
• Federated Learning with Superquantile Aggregation for Heterogeneous Data
• Anomaly Search with Multiple Plays under Delay and Switching Costs
• Robust Online Learning over Networks
• Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach
• Learning in Restless Bandits under Exogenous Global Markov Process
• Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Framework
• A Stochastic Operator Framework for Optimization and Learning with Sub-Weibull Errors
• Online Distributed Learning with Quantized Finite-Time Coordination
• Sparse Training for Federated Learning with Regularized Error Correction
• Device Scheduling and Update Aggregation Policies for Asynchronous Federated Learning
• Frequency Modulation Aggregation for Federated Learning
• Noise-Robust and Resource-Efficient ADMM-based Federated Learning
• Asynchronous Distributed Learning with Quantized Finite-Time Coordination